-- Logs begin at Wed 2022-10-19 01:38:51 UTC, end at Wed 2022-10-19 02:50:25 UTC. -- Oct 19 02:10:58.017916 dv-cinder-861491-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 19 02:10:58.025453 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 19 02:10:58.027486 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 19 02:10:58 2022] *** Oct 19 02:10:58.027486 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 19 02:10:58.027486 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 19 02:10:58.027486 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: nodename: dv-cinder-861491-4-iscsi Oct 19 02:10:58.027486 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: machine: x86_64 Oct 19 02:10:58.027486 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: clock source: unix Oct 19 02:10:58.027486 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: pcre jit disabled Oct 19 02:10:58.027974 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: detected number of CPU cores: 4 Oct 19 02:10:58.027974 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: current working directory: / Oct 19 02:10:58.027974 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: detected binary path: /usr/bin/uwsgi-core Oct 19 02:10:58.027974 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: your processes number limit is 47680 Oct 19 02:10:58.027974 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: your memory page size is 4096 bytes Oct 19 02:10:58.027974 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: detected max file descriptor number: 2048 Oct 19 02:10:58.027974 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: lock engine: pthread robust mutexes Oct 19 02:10:58.027974 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: thunder lock: enabled Oct 19 02:10:58.027974 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 19 02:10:58.027974 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 19 02:10:58.068699 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: Python main interpreter initialized at 0x555ab7e379c0 Oct 19 02:10:58.068699 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: python threads support enabled Oct 19 02:10:58.068699 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: your server socket listen backlog is limited to 100 connections Oct 19 02:10:58.068699 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: your mercy for graceful operations on workers is 90 seconds Oct 19 02:10:58.069088 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: mapped 403077 bytes (393 KB) for 2 cores Oct 19 02:10:58.069165 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: *** Operational MODE: preforking *** Oct 19 02:10:58.069437 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: *** uWSGI is running in multiple interpreter mode *** Oct 19 02:10:58.069437 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: spawned uWSGI master process (pid: 81602) Oct 19 02:10:58.069642 dv-cinder-861491-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 19 02:10:58.071646 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: spawned uWSGI worker 1 (pid: 81615, cores: 1) Oct 19 02:10:58.071646 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: spawned uWSGI worker 2 (pid: 81616, cores: 1) Oct 19 02:10:58.071646 dv-cinder-861491-4-iscsi devstack@keystone.service[81602]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 19 02:11:02.366826 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 02:11:02.366826 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: from cryptography.utils import int_from_bytes Oct 19 02:11:02.367572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 02:11:02.367572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: from cryptography.utils import int_from_bytes Oct 19 02:11:02.367996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 02:11:02.367996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: from cryptography.utils import int_from_bytes Oct 19 02:11:02.368445 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 02:11:02.368445 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: from cryptography.utils import int_from_bytes Oct 19 02:11:02.416909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 19 02:11:02.417099 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 19 02:11:02.417292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] command line args: [] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 19 02:11:02.417558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 19 02:11:02.417732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ================================================================================ {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 19 02:11:02.417982 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] admin_token = **** {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.418249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] config_dir = [] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.418336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 19 02:11:02.418436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.418523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 19 02:11:02.418601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] config_source = [] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.418663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] command line args: [] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 19 02:11:02.418786 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] debug = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.418920 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 19 02:11:02.418999 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.419215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ================================================================================ {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 19 02:11:02.419339 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.419415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] admin_token = **** {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.419480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.419598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] config_dir = [] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.419732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.419825 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.419959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] config_source = [] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.420084 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.420188 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] debug = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.420298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] list_limit = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.420408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.420537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] log_config_append = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.420537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.420730 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.420833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] log_dir = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.420936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.421040 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] log_file = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.421145 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.421231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.421356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.421433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.421555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] list_limit = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.421643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.421716 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] log_config_append = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.421716 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.421908 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.421998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] log_dir = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.422116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.422206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] log_file = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.422285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.422414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.422498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.422576 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.422701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.422785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.422857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.422972 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.423049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.423164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.423249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.423322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.423440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.423516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.423653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.423758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.423892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.424009 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.424138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.424258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.424424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.424615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.424737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.424852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] publish_errors = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.424966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.425088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.425204 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.425326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.425448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.425564 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.425677 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.425845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.425963 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.426098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] publish_errors = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.426214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] standard_threads = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.426342 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.426478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.426601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.426717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.426837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.426964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.426964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] use_journal = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.427190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.427304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] use_json = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.427419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.427540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] use_stderr = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.427666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] standard_threads = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.427789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] use_syslog = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.427905 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.428026 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.428142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.428263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.428390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.428507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.428622 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] use_journal = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.428742 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.428857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] use_json = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.428857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] use_stderr = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.429147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.429266 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] use_syslog = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.429387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.429510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 02:11:02.429625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.429746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.429873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.429990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.430149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.430273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.430389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.430513 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.430632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.430746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.430861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.430981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.431097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.431220 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.431336 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.431451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.431572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.431689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.431803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.431936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.432061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.432178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.432296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.432411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.432525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.432648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.432765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.432879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.432994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.433115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.433231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.433357 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.433475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.433589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.433704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.433825 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.433940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.434094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.434221 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.434343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.434459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.434579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.434700 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.434889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.435007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.435128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.435251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.435366 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.435481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.435605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.435723 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.435848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.436076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.436076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.436313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.436313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.436545 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.436661 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.436775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.436890 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.437004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.437004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.437230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.437345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.437459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.437573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.437687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.437810 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.437929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.438078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.438197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.438312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.438423 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.438522 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.438624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.438729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.438827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.438964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.439080 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.439180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.439278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.439384 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.439498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.439614 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.439737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.439863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.439983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.440098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.440220 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.440342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.440471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.440597 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.440721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.440850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.440973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.441091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.441207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.441322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.441446 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.441569 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.441685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.441809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.441926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.442073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.442194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.442310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.442433 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.442549 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.442671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.442800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.442918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.443034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.443149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.443265 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.443392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.443512 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.443637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.443755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.443871 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.443986 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.444101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.444229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.444353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.444470 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.444585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.444706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.444831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.444956 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.445087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.445205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.445327 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.445445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.445560 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.445675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.445799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.445922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.446038 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.446176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.446293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.446408 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.446601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.446721 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.446838 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.446954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.447075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.447198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.447324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.447443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.447559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.447674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.447796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.447919 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.448162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.448299 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.448409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.448515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.448630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.448746 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.448861 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.448989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.449107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.449223 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.449339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.449456 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.449590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.449717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.449836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.449952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.450098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.450220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.450336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.450460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.450579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.450695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.450810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.450926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.451526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.451659 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.451777 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.451893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.451893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.452109 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.452225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.452352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.452469 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.452586 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.452708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.452831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.452948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.453072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.453198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.453314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.453434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.453550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.453666 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.453790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.453914 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.454031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.454031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.454317 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.454441 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.454567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.454692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.454809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] auth.external = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.454809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.455033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.455033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.455269 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.455419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.455551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.455670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.455796 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] auth.password = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.455796 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] auth.token = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.456023 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.456147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.456273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.456392 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.456509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] auth.external = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.456626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.456742 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.456860 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.456985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.457111 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.457229 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.457346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.457463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] auth.password = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.457587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.457706 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] auth.token = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.457824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] credential.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.457941 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.458087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.458208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.458338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.458458 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.458575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.458691 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.458807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.458932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.459059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.459178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.459294 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.459411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.459528 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.459892 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] credential.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.460019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.460138 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.460255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.460371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.460487 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.460603 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.460727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.460845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.460963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.461079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.461079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.461297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.461297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.461601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.461719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.461835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.461950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] federation.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.462086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.462218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.462338 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.462455 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.462571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.462687 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.462810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.462937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.463056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.463173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.463290 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.463407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.463523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.463649 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.463768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] federation.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.463884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.464001 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.464118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.464235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.464359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.464478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.464595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.464711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.464827 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.464943 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.465105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.465228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.465351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.465476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.465594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.465711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.465839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.465966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.466151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.466273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.466389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.466505 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.466637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.467312 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.467440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.467557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.467674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.467791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.467917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.468036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.468153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.468269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.468387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.468503 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.468629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.468748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.468865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.468981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.469097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.469222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.469342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.469459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.469576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.469693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.469809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.469933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.470067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.470186 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.470303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.470420 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.470537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.470665 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.470785 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.470902 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.471018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.471018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.471237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.471362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.471483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.471599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.471722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.471722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.471942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.472068 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.472187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.472303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.472420 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.472538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.472655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.472779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.472898 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.473015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.473131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.473248 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.473364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.473489 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.473609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.473609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.473827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.473943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.474104 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.474237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.474358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.474474 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.474600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.474719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.474836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.474964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.475083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.475200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.475317 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.475433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.475549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.475675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.475794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.475914 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.476032 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.476148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.476264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.476389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.476508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.476624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.476742 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.476858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.476974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.477098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.477217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.477333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.477448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.477564 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.477680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.477806 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.477925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.478058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.478178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.478294 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.478410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.478539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.478658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.478774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.478891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.479006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.479122 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.479253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.479372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.479489 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.479605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.479721 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.479838 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.479964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.480082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.480198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.480315 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.480431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.480555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.480674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.480790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.480907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.481023 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.481140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.481264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.481383 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.481500 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.481616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.481732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.481848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.481974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.482118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.482237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.482354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.482470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.482587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.482716 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.482836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.482953 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.483071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.483187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.483303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.483303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.483532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.483649 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.483765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.483881 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.483998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.484123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.484242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.484359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.484476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.484592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.484708 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.484900 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.485020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.485137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.485254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.485371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.485487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.485611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.485731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.485847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.485967 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.486109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.486229 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.486354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] resource.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.486473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.486590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.486706 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.486823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.486939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.487064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.487183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.487300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.487416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.487532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.487648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.487773 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.487891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.488007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.488123 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.488240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.488356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.488480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.488599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] resource.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.488716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] role.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.488832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.488948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] role.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.489065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.489189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.489309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.489426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.489543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.489660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.489776 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.489928 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.490067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.490187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.490304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.490422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.490538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.490667 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.490787 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.490904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.491020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] role.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.491136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.491252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] role.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.491377 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.491495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.491612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.491727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.491843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.491960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.492087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.492206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.492322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.492439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.492439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.492666 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.492784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.492901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.493017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.493133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.493250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.493374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.493495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.493611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.493727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.493844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.493844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.494118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.494241 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.494359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.494475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.494593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.494709 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.494833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.494959 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.495077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.495193 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.495309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.495432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.495558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.495677 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.495793 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.495913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.496030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.496146 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.496270 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.496389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.496506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.496623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.496745 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.496863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.496863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] token.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.497093 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.497210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.497327 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.497327 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.497544 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.497669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.497788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.497904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.498020 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.498156 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.498273 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.498273 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.498506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.498623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.498740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.498740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.498967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.499086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.499202 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.499318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] token.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.499434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.499550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.499674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.499674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.499896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.499896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.500114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.500229 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.500354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.500354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.500583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.500583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.500835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.500951 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.501068 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.501183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.501308 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.connection = **** {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.501426 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.501542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.501658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.501774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.501891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.502016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.502153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.502271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.502387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.502503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.502503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.502733 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.502853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.502970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.503096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.503222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.503340 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.503456 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.503647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.503768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.connection = **** {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.503885 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.504001 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.504118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.504235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.504363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.504483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.504599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.504715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.504832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.504948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505425 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81615) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 19 02:11:02.505906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.505906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.506930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 02:11:02.506930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81616) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 19 02:11:02.530016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.530262 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.530485 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.532042 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.532277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.532504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.538719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.540745 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.566261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.566490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.568587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.568820 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.620028 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.620576 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.668268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.668498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.669948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.670207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.823073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.823308 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.823526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.823742 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.823893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.823980 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.824127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.824210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.824349 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.824566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.824783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.824997 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.847419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.847631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.847718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.847856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.847938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.848078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.848161 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.848296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.848380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.848508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.848590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.848719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.848810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.848951 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.859753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.860508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.872314 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.872504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.872590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.872728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.872814 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.873472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.873557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.873688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.873897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.873978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.874131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81615) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.874228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.874395 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81616) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 19 02:11:02.891145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.891353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 19 02:11:02.891692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.891892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 19 02:11:02.892164 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.892244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.892372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 19 02:11:02.892452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.892657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.892746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.892900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 19 02:11:02.892981 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.893178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.893440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.893703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.893960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.902407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.902684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.902951 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.903235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.903681 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.903962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.904355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.904646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.910213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.911643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.913106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.913387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.913665 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.913942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.914247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.914506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.914705 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.914787 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.914990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.915070 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.915272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.915354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.915536 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.915615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.915799 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.915887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.916065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.916336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.916608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.916879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.917153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.917421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.939050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.940503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.940782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.940862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.942272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.942558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.945457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.945735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.946013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.947239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.947524 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.947805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.952879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.953159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.954724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.955007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.958727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.959022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.959308 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.959844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.960123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.960396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.960666 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.960842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.960923 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.961140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.961431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.961718 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.961991 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.962289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.962561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.962831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.976410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.976638 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.978301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.978581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.980475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.982430 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.984594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.984963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.988106 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.988606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:02.990214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.993955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.995095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:02.997208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.009759 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.009759 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.009759 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.010098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.010182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.010182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.010351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.010351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.010351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.010351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.010351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.010351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.015633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.015918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.016187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.016450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.019880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.020169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.020434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.020694 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.023279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.024807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.026343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.026619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.027605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.029170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.029459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.029741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.030025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.030330 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.030725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.030821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.031001 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.033748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.034065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.034393 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.034713 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.035139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.041915 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.042218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.042497 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.042793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.043168 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.043452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.043729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.046934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.047223 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.047506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.047821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.048176 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.048474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.048755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.059564 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.063024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.064761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.065897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.067511 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.067714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.069076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.069358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.069797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.070667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.072113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.073703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.073987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.074495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.076149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.077611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.080512 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.080904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.080988 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.081258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.081504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.082522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.085664 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.086078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.086376 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.086654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.089632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.089920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.090231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.094549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.094839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.095120 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.097110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81615) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.097607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.097696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.097924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.098216 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.098490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.098760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.099042 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.099309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.099685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.099865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.100138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.100426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.100695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81615) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.102110 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81616) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 19 02:11:03.102400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.102685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.102959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.103228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.103589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.103865 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.104236 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.104509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.104800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.105075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.105344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.106140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.106140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81616) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 19 02:11:03.135694 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555ab7e379c0 pid: 81615 (default app) Oct 19 02:11:03.138952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58116d5-1159-4318-850f-97a045007bca None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:03.139240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58116d5-1159-4318-850f-97a045007bca None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:03.139515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58116d5-1159-4318-850f-97a045007bca None None] PATH_INFO: `/v3` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:03.141305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555ab7e379c0 pid: 81616 (default app) Oct 19 02:11:03.143664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Wed Oct 19 02:11:03 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:10.817475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e012279-c31d-4927-8a92-2cc1c8019295 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:10.818015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e012279-c31d-4927-8a92-2cc1c8019295 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:10.818015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e012279-c31d-4927-8a92-2cc1c8019295 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:10.822290 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:10.827979 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6ebe-6054-4016-8ea8-439b77053cff None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:10.828460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6ebe-6054-4016-8ea8-439b77053cff None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:10.828556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6ebe-6054-4016-8ea8-439b77053cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:10.898234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: INFO dbcounter [None req-ee1f6ebe-6054-4016-8ea8-439b77053cff None None] Registered counter for database keystone Oct 19 02:11:10.919331 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG oslo_db.sqlalchemy.engines [None req-ee1f6ebe-6054-4016-8ea8-439b77053cff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81615) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 02:11:10.925518 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writer thread running {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 02:11:11.107824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG passlib.handlers.bcrypt [None req-ee1f6ebe-6054-4016-8ea8-439b77053cff None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81615) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 19 02:11:11.108207 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG passlib.handlers.bcrypt [None req-ee1f6ebe-6054-4016-8ea8-439b77053cff None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81615) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 19 02:11:11.165841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-ee1f6ebe-6054-4016-8ea8-439b77053cff None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:11.167463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ee1f6ebe-6054-4016-8ea8-439b77053cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:11.294133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 19 02:11:11.741791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e012279-c31d-4927-8a92-2cc1c8019295 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:11.810743 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: INFO dbcounter [None req-8e012279-c31d-4927-8a92-2cc1c8019295 None None] Registered counter for database keystone Oct 19 02:11:11.829499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG oslo_db.sqlalchemy.engines [None req-8e012279-c31d-4927-8a92-2cc1c8019295 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81616) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 02:11:11.836040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writer thread running {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 02:11:11.952977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e012279-c31d-4927-8a92-2cc1c8019295 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:11.957358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd6b98-3ed5-477e-89c4-6004c0309c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:11.957983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd6b98-3ed5-477e-89c4-6004c0309c3d admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:11.958305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd6b98-3ed5-477e-89c4-6004c0309c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:11.958579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd6b98-3ed5-477e-89c4-6004c0309c3d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:11.967423 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd6b98-3ed5-477e-89c4-6004c0309c3d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:12.019793 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: WARNING keystone.common.rbac_enforcer.enforcer [None req-13bd6b98-3ed5-477e-89c4-6004c0309c3d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 19 02:11:12.020088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd6b98-3ed5-477e-89c4-6004c0309c3d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:12.029750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Oct 19 02:11:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:11:12.035779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f6ebe-6054-4016-8ea8-439b77053cff None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:12.053077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f6ebe-6054-4016-8ea8-439b77053cff None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:12.057463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279572b-27aa-440d-930c-7c2f3e833b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:12.058587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279572b-27aa-440d-930c-7c2f3e833b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:12.058871 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279572b-27aa-440d-930c-7c2f3e833b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:12.059144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279572b-27aa-440d-930c-7c2f3e833b49 admin admin] PATH_INFO: `/v3/projects/984205d302cd4aa3b16c6c1e0788c92c` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:12.060731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279572b-27aa-440d-930c-7c2f3e833b49 admin admin] RBAC: Authorizing `identity:get_project(project_id=984205d302cd4aa3b16c6c1e0788c92c)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:12.114803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8279572b-27aa-440d-930c-7c2f3e833b49 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 19 02:11:12.115164 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8279572b-27aa-440d-930c-7c2f3e833b49 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:12.119705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 19 02:11:12 2022] GET /identity/v3/projects/984205d302cd4aa3b16c6c1e0788c92c => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:14.224779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242a331-7e62-4898-9c6f-a83218aa6b1e None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:14.225412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242a331-7e62-4898-9c6f-a83218aa6b1e None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:14.225412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242a331-7e62-4898-9c6f-a83218aa6b1e None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:14.229463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:11:14.235483 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d93f53-7c72-4a64-a611-d91cbb6dd36a None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:14.235865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d93f53-7c72-4a64-a611-d91cbb6dd36a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:14.236034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d93f53-7c72-4a64-a611-d91cbb6dd36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:14.255661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-a5d93f53-7c72-4a64-a611-d91cbb6dd36a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:14.257178 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a5d93f53-7c72-4a64-a611-d91cbb6dd36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:14.264918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 19 02:11:14.533199 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242a331-7e62-4898-9c6f-a83218aa6b1e None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:14.545601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242a331-7e62-4898-9c6f-a83218aa6b1e None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:14.549716 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8a063-563d-4df7-8637-217d90af1483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:14.550345 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8a063-563d-4df7-8637-217d90af1483 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:14.550625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8a063-563d-4df7-8637-217d90af1483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:14.550896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8a063-563d-4df7-8637-217d90af1483 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:14.558350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8a063-563d-4df7-8637-217d90af1483 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:14.560119 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8a063-563d-4df7-8637-217d90af1483 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:14.569186 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Oct 19 02:11:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:11:14.574463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d93f53-7c72-4a64-a611-d91cbb6dd36a None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:14.583340 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d93f53-7c72-4a64-a611-d91cbb6dd36a None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:14.587877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1628d3-4a27-4f47-ae87-4893c2bbc706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:14.588497 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1628d3-4a27-4f47-ae87-4893c2bbc706 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:14.588778 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1628d3-4a27-4f47-ae87-4893c2bbc706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:14.589052 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1628d3-4a27-4f47-ae87-4893c2bbc706 admin admin] PATH_INFO: `/v3/users/88e388dc7d10482d8944dbc5c7039f97` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:14.590393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1628d3-4a27-4f47-ae87-4893c2bbc706 admin admin] RBAC: Authorizing `identity:get_user(user_id=88e388dc7d10482d8944dbc5c7039f97)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:14.592213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1628d3-4a27-4f47-ae87-4893c2bbc706 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:14.593711 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Wed Oct 19 02:11:14 2022] GET /identity/v3/users/88e388dc7d10482d8944dbc5c7039f97 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:16.768149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2bca89-5514-453b-b82a-77fb393c356b None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:16.768690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2bca89-5514-453b-b82a-77fb393c356b None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:16.768690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2bca89-5514-453b-b82a-77fb393c356b None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:16.770007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:16.775008 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56c90f-15e3-454d-8798-2cdcdf42d00f None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:16.775392 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56c90f-15e3-454d-8798-2cdcdf42d00f None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:16.775630 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa56c90f-15e3-454d-8798-2cdcdf42d00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:16.793269 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-aa56c90f-15e3-454d-8798-2cdcdf42d00f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:16.794475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-aa56c90f-15e3-454d-8798-2cdcdf42d00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:16.824297 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9656beb-6255-4cf9-8587-8708decfeeb3 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:16.824579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9656beb-6255-4cf9-8587-8708decfeeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:16.824860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9656beb-6255-4cf9-8587-8708decfeeb3 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:16.826238 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:11:16.831359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9ec4f-73f5-4cd1-995d-de3b00b42532 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:16.831641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9ec4f-73f5-4cd1-995d-de3b00b42532 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:16.831912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9ec4f-73f5-4cd1-995d-de3b00b42532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:16.872653 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG passlib.handlers.bcrypt [None req-9ee9ec4f-73f5-4cd1-995d-de3b00b42532 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81616) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 19 02:11:16.873645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG passlib.handlers.bcrypt [None req-9ee9ec4f-73f5-4cd1-995d-de3b00b42532 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81616) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 19 02:11:16.877349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:16.922967 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9ee9ec4f-73f5-4cd1-995d-de3b00b42532 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:16.924526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9ee9ec4f-73f5-4cd1-995d-de3b00b42532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:17.021025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:11:17.145730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56c90f-15e3-454d-8798-2cdcdf42d00f None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:17.171074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:17.247710 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa56c90f-15e3-454d-8798-2cdcdf42d00f None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:17.279033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd49fd6-4a32-4dc6-9881-51e22b71931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:17.280114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd49fd6-4a32-4dc6-9881-51e22b71931c None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:17.280388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd49fd6-4a32-4dc6-9881-51e22b71931c None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:17.280653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd49fd6-4a32-4dc6-9881-51e22b71931c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:17.291389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd49fd6-4a32-4dc6-9881-51e22b71931c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:17.300789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9ec4f-73f5-4cd1-995d-de3b00b42532 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-ddd49fd6-4a32-4dc6-9881-51e22b71931c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:17.300944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:17.302006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:17.303021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:11:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:17.307424 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd49fd6-4a32-4dc6-9881-51e22b71931c None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:17.387105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:17.404711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9ec4f-73f5-4cd1-995d-de3b00b42532 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:17.406156 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd49fd6-4a32-4dc6-9881-51e22b71931c None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:17.407282 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:17.439409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f5346-da8a-4835-845b-4bb9d6b094f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:17.440009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5346-da8a-4835-845b-4bb9d6b094f2 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:17.440287 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5346-da8a-4835-845b-4bb9d6b094f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:17.440557 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5346-da8a-4835-845b-4bb9d6b094f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:17.441216 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f5346-da8a-4835-845b-4bb9d6b094f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:17.441299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48dd14-df31-404f-9f99-9ae60b7dbcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:17.441562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f5346-da8a-4835-845b-4bb9d6b094f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:17.442164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48dd14-df31-404f-9f99-9ae60b7dbcb4 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:17.442444 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48dd14-df31-404f-9f99-9ae60b7dbcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:17.442714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48dd14-df31-404f-9f99-9ae60b7dbcb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:17.442966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f5346-da8a-4835-845b-4bb9d6b094f2 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:17.443665 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48dd14-df31-404f-9f99-9ae60b7dbcb4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:17.445069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48dd14-df31-404f-9f99-9ae60b7dbcb4 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:17.455154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:17.460924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f5346-da8a-4835-845b-4bb9d6b094f2 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:17.482262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.sql.core [None req-9b48dd14-df31-404f-9f99-9ae60b7dbcb4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 02:11:17.482262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 02:11:17.482262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [parameters: {'id': 'aeb386eecb1a4ae2aedb202610dec7cb', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 19 02:11:17.482262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81616) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-9b48dd14-df31-404f-9f99-9ae60b7dbcb4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application conn.query(q) Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 02:11:17.500674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result.read() Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 19 02:11:17.501585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application next(self.gen) Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application yield resource Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application next(self.gen) Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.session.rollback() Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.force_reraise() Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise self.value Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application session.commit() Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 19 02:11:17.502486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._prepare_impl() Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.session.flush() Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._flush(objects) Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application compat.raise_( Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application flush_context.execute() Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rec.execute(self) Oct 19 02:11:17.503295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 19 02:11:17.504211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application conn.query(q) Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result.read() Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 02:11:17.504940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [parameters: {'id': 'aeb386eecb1a4ae2aedb202610dec7cb', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 19 02:11:17.505817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:17.506984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:17.508015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 8/16] 10.222.0.20 () {60 vars in 1137 bytes} [Wed Oct 19 02:11:17 2022] POST /identity/v3/domains => generated 190 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:11:17.508827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48dd14-df31-404f-9f99-9ae60b7dbcb4 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:17.565554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f5346-da8a-4835-845b-4bb9d6b094f2 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:17.598928 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339913b-d14c-4f76-871b-e3c904d3d307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:17.599549 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339913b-d14c-4f76-871b-e3c904d3d307 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:17.599825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339913b-d14c-4f76-871b-e3c904d3d307 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:17.600097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339913b-d14c-4f76-871b-e3c904d3d307 None admin] PATH_INFO: `/v3/users/88e388dc7d10482d8944dbc5c7039f97` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:17.601227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339913b-d14c-4f76-871b-e3c904d3d307 None admin] RBAC: Authorizing `identity:get_user(user_id=88e388dc7d10482d8944dbc5c7039f97)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:17.602721 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339913b-d14c-4f76-871b-e3c904d3d307 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:17.604264 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 9/17] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:17 2022] GET /identity/v3/users/88e388dc7d10482d8944dbc5c7039f97 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:17.607567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48dd14-df31-404f-9f99-9ae60b7dbcb4 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:17.609644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339913b-d14c-4f76-871b-e3c904d3d307 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:17.623517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:17.640212 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d510858-3478-4cac-a4ba-773680b3ae19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:17.641092 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510858-3478-4cac-a4ba-773680b3ae19 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:17.641363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510858-3478-4cac-a4ba-773680b3ae19 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:17.641623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d510858-3478-4cac-a4ba-773680b3ae19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:17.659673 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d510858-3478-4cac-a4ba-773680b3ae19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:17.661124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d510858-3478-4cac-a4ba-773680b3ae19 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-9d510858-3478-4cac-a4ba-773680b3ae19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:11:17.676877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:17.682252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:11:17.683084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:11:17.683084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:11:17.683084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:17.683084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 9/18] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:17.684459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d510858-3478-4cac-a4ba-773680b3ae19 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:17.706899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339913b-d14c-4f76-871b-e3c904d3d307 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:17.707771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:17.743607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e05082-7274-4c61-96b2-e43fb48a3920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:17.743607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05082-7274-4c61-96b2-e43fb48a3920 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:17.743607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05082-7274-4c61-96b2-e43fb48a3920 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:17.744362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e05082-7274-4c61-96b2-e43fb48a3920 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:17.745188 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e05082-7274-4c61-96b2-e43fb48a3920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:17.746667 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e05082-7274-4c61-96b2-e43fb48a3920 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:17.748451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 10/19] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:17.753737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e05082-7274-4c61-96b2-e43fb48a3920 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:17.786863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d510858-3478-4cac-a4ba-773680b3ae19 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:17.820008 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdeebf6-06d1-4b17-8023-8169b256f0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:17.820549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeebf6-06d1-4b17-8023-8169b256f0db None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:17.820832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeebf6-06d1-4b17-8023-8169b256f0db None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:17.821106 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdeebf6-06d1-4b17-8023-8169b256f0db None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:17.821746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdeebf6-06d1-4b17-8023-8169b256f0db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:17.822075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdeebf6-06d1-4b17-8023-8169b256f0db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:17.823440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdeebf6-06d1-4b17-8023-8169b256f0db None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:17.831978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:17.840867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 10/20] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:17.851223 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e05082-7274-4c61-96b2-e43fb48a3920 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:17.888129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d6205-7a6c-4745-95f1-05e73c675b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:17.889354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d6205-7a6c-4745-95f1-05e73c675b2a None admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:17.889626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d6205-7a6c-4745-95f1-05e73c675b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:17.889893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d6205-7a6c-4745-95f1-05e73c675b2a None admin] PATH_INFO: `/v3/domains/default/users/88e388dc7d10482d8944dbc5c7039f97/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:17.891965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d6205-7a6c-4745-95f1-05e73c675b2a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=88e388dc7d10482d8944dbc5c7039f97, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:17.893586 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d6205-7a6c-4745-95f1-05e73c675b2a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:18.129790 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 11/21] 10.222.0.20 () {58 vars in 1361 bytes} [Wed Oct 19 02:11:17 2022] PUT /identity/v3/domains/default/users/88e388dc7d10482d8944dbc5c7039f97/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:11:20.298278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2378a-28fc-4692-ae9f-e4bf272d5045 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:20.298766 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2378a-28fc-4692-ae9f-e4bf272d5045 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:20.298862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2378a-28fc-4692-ae9f-e4bf272d5045 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:20.300298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 11/22] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:20.306344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219fd20-2bae-478b-af14-3cd54490dbfd None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:20.306747 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219fd20-2bae-478b-af14-3cd54490dbfd None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:20.306935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219fd20-2bae-478b-af14-3cd54490dbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:20.335693 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-f219fd20-2bae-478b-af14-3cd54490dbfd None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:20.337236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f219fd20-2bae-478b-af14-3cd54490dbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:20.425618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 12/23] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:20.655034 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eac0b-dcbe-47b6-a9d4-8976ad898189 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:20.655697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eac0b-dcbe-47b6-a9d4-8976ad898189 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:20.656170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16eac0b-dcbe-47b6-a9d4-8976ad898189 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:20.657718 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 12/24] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:20.674733 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb4fb8-d725-4b2b-9f98-289e221d3104 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:20.675011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb4fb8-d725-4b2b-9f98-289e221d3104 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:20.675292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb4fb8-d725-4b2b-9f98-289e221d3104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:20.699603 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8efb4fb8-d725-4b2b-9f98-289e221d3104 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:20.700773 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8efb4fb8-d725-4b2b-9f98-289e221d3104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:20.725403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826793f-5f12-4113-9cf1-fa42658aa957 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:20.726026 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826793f-5f12-4113-9cf1-fa42658aa957 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:20.726536 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826793f-5f12-4113-9cf1-fa42658aa957 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:20.728437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 13/25] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:11:20.733934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaab781-73f1-4f84-8ae1-a44bad25cb77 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:20.734568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaab781-73f1-4f84-8ae1-a44bad25cb77 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:20.735030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaab781-73f1-4f84-8ae1-a44bad25cb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:20.771063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ddaab781-73f1-4f84-8ae1-a44bad25cb77 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:20.772600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ddaab781-73f1-4f84-8ae1-a44bad25cb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:20.805302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:20.814373 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 13/26] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:20.891309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 14/27] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:20.934976 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb4fb8-d725-4b2b-9f98-289e221d3104 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:21.077261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb4fb8-d725-4b2b-9f98-289e221d3104 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:21.153380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6708e-377b-47c1-a95a-d195be817994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:21.154034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6708e-377b-47c1-a95a-d195be817994 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:21.155962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6708e-377b-47c1-a95a-d195be817994 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:21.155962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6708e-377b-47c1-a95a-d195be817994 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:21.173465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaab781-73f1-4f84-8ae1-a44bad25cb77 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:21.174243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6708e-377b-47c1-a95a-d195be817994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:21.175962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6708e-377b-47c1-a95a-d195be817994 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-d0b6708e-377b-47c1-a95a-d195be817994 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:11:21.204538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:21.205457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:11:21.206247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:11:21.206247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:11:21.206247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:21.206247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 14/28] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:21.213924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6708e-377b-47c1-a95a-d195be817994 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:21.214515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:21.330445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:21.392094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6708e-377b-47c1-a95a-d195be817994 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:21.393142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaab781-73f1-4f84-8ae1-a44bad25cb77 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:21.429305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b53916-30b5-440c-9e51-6ccb80c3198e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:21.429992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b53916-30b5-440c-9e51-6ccb80c3198e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:21.430308 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b53916-30b5-440c-9e51-6ccb80c3198e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:21.430581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b53916-30b5-440c-9e51-6ccb80c3198e None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:21.431230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b53916-30b5-440c-9e51-6ccb80c3198e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:21.431529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b53916-30b5-440c-9e51-6ccb80c3198e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:21.432995 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b53916-30b5-440c-9e51-6ccb80c3198e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:21.436182 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9945487-923d-4aa6-9bb4-4508145876d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:21.437043 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9945487-923d-4aa6-9bb4-4508145876d7 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:21.437535 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9945487-923d-4aa6-9bb4-4508145876d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:21.438421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9945487-923d-4aa6-9bb4-4508145876d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:21.439132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9945487-923d-4aa6-9bb4-4508145876d7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:21.440518 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9945487-923d-4aa6-9bb4-4508145876d7 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:21.461344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 15/29] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:21.463124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3a2c2-e189-447a-9173-e9b596a16800 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:21.463616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3a2c2-e189-447a-9173-e9b596a16800 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:21.464145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3a2c2-e189-447a-9173-e9b596a16800 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:21.465557 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 16/30] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:21.466587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3a2c2-e189-447a-9173-e9b596a16800 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:21.583004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3a2c2-e189-447a-9173-e9b596a16800 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:21.601009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:21.625205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e25d15-5fe4-4016-a93b-ed569dfe2f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:21.626249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e25d15-5fe4-4016-a93b-ed569dfe2f35 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:21.626730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e25d15-5fe4-4016-a93b-ed569dfe2f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:21.627246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e25d15-5fe4-4016-a93b-ed569dfe2f35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:21.639351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e25d15-5fe4-4016-a93b-ed569dfe2f35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-50e25d15-5fe4-4016-a93b-ed569dfe2f35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:21.644892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:21.645728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:21.646509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:21.648691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 17/31] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:11:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:21.650974 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e25d15-5fe4-4016-a93b-ed569dfe2f35 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:21.665074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 15/32] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Oct 19 02:11:21 2022] POST /identity/v3/roles => generated 219 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:11:21.671399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa7620-9782-450c-b0ef-df89e4b7ee41 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:21.671715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa7620-9782-450c-b0ef-df89e4b7ee41 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:21.671952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa7620-9782-450c-b0ef-df89e4b7ee41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:21.709560 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-f5fa7620-9782-450c-b0ef-df89e4b7ee41 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:21.711482 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f5fa7620-9782-450c-b0ef-df89e4b7ee41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:21.858553 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:21.868391 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 16/33] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:21.871064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa7620-9782-450c-b0ef-df89e4b7ee41 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:22.011568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa7620-9782-450c-b0ef-df89e4b7ee41 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:22.024504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e25d15-5fe4-4016-a93b-ed569dfe2f35 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:22.059821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e6a95-e82e-493b-b01a-4778f9d22b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:22.060608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e6a95-e82e-493b-b01a-4778f9d22b53 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:22.061204 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e6a95-e82e-493b-b01a-4778f9d22b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:22.061559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e6a95-e82e-493b-b01a-4778f9d22b53 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:22.062751 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e6a95-e82e-493b-b01a-4778f9d22b53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:22.063135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e6a95-e82e-493b-b01a-4778f9d22b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:22.065236 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e6a95-e82e-493b-b01a-4778f9d22b53 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:22.067767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814192c8-7b09-4a17-b9ce-30c0f4c6f598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:22.068594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814192c8-7b09-4a17-b9ce-30c0f4c6f598 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:22.069052 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814192c8-7b09-4a17-b9ce-30c0f4c6f598 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:22.069540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814192c8-7b09-4a17-b9ce-30c0f4c6f598 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:22.070486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814192c8-7b09-4a17-b9ce-30c0f4c6f598 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:22.072172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814192c8-7b09-4a17-b9ce-30c0f4c6f598 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:22.079114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 17/34] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:22.085084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e83d3-a533-4326-916e-304c13be1d7c None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:22.085649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e83d3-a533-4326-916e-304c13be1d7c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:22.085984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e83d3-a533-4326-916e-304c13be1d7c None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:22.087430 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 18/35] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:22.089596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e83d3-a533-4326-916e-304c13be1d7c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:22.168258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:22.190531 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 18/36] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:11:21 2022] POST /identity/v3/projects => generated 301 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:22.192417 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092420f-eb67-46d5-aa80-5418a875a5ff None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:22.193013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092420f-eb67-46d5-aa80-5418a875a5ff None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:22.193520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092420f-eb67-46d5-aa80-5418a875a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:22.220175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e83d3-a533-4326-916e-304c13be1d7c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:22.223305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8092420f-eb67-46d5-aa80-5418a875a5ff None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:22.225897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8092420f-eb67-46d5-aa80-5418a875a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:22.254092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:22.267715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e250082-1869-4c85-8692-92750451b661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:22.268391 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250082-1869-4c85-8692-92750451b661 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:22.268740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250082-1869-4c85-8692-92750451b661 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:22.269027 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250082-1869-4c85-8692-92750451b661 None admin] PATH_INFO: `/v3/users/88e388dc7d10482d8944dbc5c7039f97` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:22.270620 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e250082-1869-4c85-8692-92750451b661 None admin] RBAC: Authorizing `identity:get_user(user_id=88e388dc7d10482d8944dbc5c7039f97)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:22.272268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e250082-1869-4c85-8692-92750451b661 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:22.274268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 19/37] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:22 2022] GET /identity/v3/users/88e388dc7d10482d8944dbc5c7039f97 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:22.274859 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e250082-1869-4c85-8692-92750451b661 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:22.348977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 19/38] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:22.350142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092420f-eb67-46d5-aa80-5418a875a5ff None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:22.428688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e250082-1869-4c85-8692-92750451b661 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:22.465439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:22.470881 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac8d69-97d9-49ab-98c7-15de7101cbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:22.471572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac8d69-97d9-49ab-98c7-15de7101cbf8 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:22.471851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac8d69-97d9-49ab-98c7-15de7101cbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:22.472131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac8d69-97d9-49ab-98c7-15de7101cbf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:22.476574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac8d69-97d9-49ab-98c7-15de7101cbf8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:22.476574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac8d69-97d9-49ab-98c7-15de7101cbf8 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:22.479496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092420f-eb67-46d5-aa80-5418a875a5ff None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:22.517460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fee18d-cfad-4fc6-af58-eeecab3ad68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:22.518114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fee18d-cfad-4fc6-af58-eeecab3ad68d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:22.518409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fee18d-cfad-4fc6-af58-eeecab3ad68d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:22.518675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fee18d-cfad-4fc6-af58-eeecab3ad68d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:22.519810 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fee18d-cfad-4fc6-af58-eeecab3ad68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:22.521274 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fee18d-cfad-4fc6-af58-eeecab3ad68d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:22.523088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 20/39] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:22.524118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b8f7b-ac59-4615-a8be-589ecca7cafb None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:22.524410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b8f7b-ac59-4615-a8be-589ecca7cafb None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:22.524689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b8f7b-ac59-4615-a8be-589ecca7cafb None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:22.525922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 21/40] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:22.531425 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c93fa-96a7-4604-b4a3-8a9c9a1b7768 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:22.531704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c93fa-96a7-4604-b4a3-8a9c9a1b7768 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:22.531980 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c93fa-96a7-4604-b4a3-8a9c9a1b7768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:22.540822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 20/41] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Oct 19 02:11:22 2022] POST /identity/v3/roles => generated 225 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:22.541687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac8d69-97d9-49ab-98c7-15de7101cbf8 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:22.559530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-f53c93fa-96a7-4604-b4a3-8a9c9a1b7768 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:22.563061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f53c93fa-96a7-4604-b4a3-8a9c9a1b7768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:22.589800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:22.663191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 22/42] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 19 02:11:22.667628 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c93fa-96a7-4604-b4a3-8a9c9a1b7768 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:22.673843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac8d69-97d9-49ab-98c7-15de7101cbf8 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:22.724840 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec7306-2a89-4b5d-95ee-8af93506cd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:22.725388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec7306-2a89-4b5d-95ee-8af93506cd0a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:22.725674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec7306-2a89-4b5d-95ee-8af93506cd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:22.725952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec7306-2a89-4b5d-95ee-8af93506cd0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:22.726730 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec7306-2a89-4b5d-95ee-8af93506cd0a None admin] RBAC: Adding query filter params (role.id=d5070bb151d343ca8de32abe373f4a73, scope.domain.id=default, user.id=88e388dc7d10482d8944dbc5c7039f97) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:22.727044 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec7306-2a89-4b5d-95ee-8af93506cd0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:22.728549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec7306-2a89-4b5d-95ee-8af93506cd0a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:22.738998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 21/43] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Oct 19 02:11:22 2022] GET /identity/v3/role_assignments?user.id=88e388dc7d10482d8944dbc5c7039f97&scope.domain.id=default&role.id=d5070bb151d343ca8de32abe373f4a73 => generated 530 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:22.794106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c93fa-96a7-4604-b4a3-8a9c9a1b7768 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:22.834358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6e827-3846-4d87-b997-a8671aebb6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:22.835054 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6e827-3846-4d87-b997-a8671aebb6ff None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:22.835608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6e827-3846-4d87-b997-a8671aebb6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:22.835886 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6e827-3846-4d87-b997-a8671aebb6ff None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:22.836583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6e827-3846-4d87-b997-a8671aebb6ff None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:22.838014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6e827-3846-4d87-b997-a8671aebb6ff None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:22.943722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec7306-2a89-4b5d-95ee-8af93506cd0a None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:22.951852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 23/44] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Oct 19 02:11:22 2022] POST /identity/v3/roles => generated 223 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:23.010356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:23.057509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec7306-2a89-4b5d-95ee-8af93506cd0a None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:23.092806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607449f-0a80-4ed2-940b-61428d0e3803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:23.093462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607449f-0a80-4ed2-940b-61428d0e3803 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:23.093743 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607449f-0a80-4ed2-940b-61428d0e3803 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:23.094016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607449f-0a80-4ed2-940b-61428d0e3803 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:23.095211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607449f-0a80-4ed2-940b-61428d0e3803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:23.096695 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607449f-0a80-4ed2-940b-61428d0e3803 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:23.098583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 22/45] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:23.104188 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6e827-3846-4d87-b997-a8671aebb6ff None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:23.120579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:23.209656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6e827-3846-4d87-b997-a8671aebb6ff None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:23.243239 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d025f4c-1fa2-4f16-9b3d-1b36f8fff564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:23.243843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d025f4c-1fa2-4f16-9b3d-1b36f8fff564 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:23.244128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d025f4c-1fa2-4f16-9b3d-1b36f8fff564 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:23.244406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d025f4c-1fa2-4f16-9b3d-1b36f8fff564 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:23.245059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d025f4c-1fa2-4f16-9b3d-1b36f8fff564 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:23.246629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d025f4c-1fa2-4f16-9b3d-1b36f8fff564 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:23.362654 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 24/46] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:11:23 2022] POST /identity/v3/projects => generated 312 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:25.490535 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c025b-11ce-468e-b32a-dae76a76eae2 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:25.491037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c025b-11ce-468e-b32a-dae76a76eae2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:25.491125 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c025b-11ce-468e-b32a-dae76a76eae2 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:25.492436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 23/47] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:25.497507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16592f71-d1c9-43d8-8794-e7ec9307a3bf None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:25.498460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16592f71-d1c9-43d8-8794-e7ec9307a3bf None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:25.498588 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16592f71-d1c9-43d8-8794-e7ec9307a3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:25.517007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-16592f71-d1c9-43d8-8794-e7ec9307a3bf None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:25.518219 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-16592f71-d1c9-43d8-8794-e7ec9307a3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:25.573993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:25.581640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 25/48] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:25.850635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c025b-11ce-468e-b32a-dae76a76eae2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:25.949819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c025b-11ce-468e-b32a-dae76a76eae2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:25.971470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:25.982954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa971f-0cfa-4d03-8630-2eb44e7bf5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:25.983584 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa971f-0cfa-4d03-8630-2eb44e7bf5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:25.983866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa971f-0cfa-4d03-8630-2eb44e7bf5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:25.984140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa971f-0cfa-4d03-8630-2eb44e7bf5c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:25.985269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa971f-0cfa-4d03-8630-2eb44e7bf5c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:25.986753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa971f-0cfa-4d03-8630-2eb44e7bf5c8 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:25.988283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 24/49] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:25.994154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16592f71-d1c9-43d8-8794-e7ec9307a3bf None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:26.087018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16592f71-d1c9-43d8-8794-e7ec9307a3bf None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:26.122224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ece37f-5541-4ae8-8a11-876ea3bc4d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:26.122822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ece37f-5541-4ae8-8a11-876ea3bc4d25 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:26.123124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ece37f-5541-4ae8-8a11-876ea3bc4d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:26.123396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ece37f-5541-4ae8-8a11-876ea3bc4d25 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:26.124038 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ece37f-5541-4ae8-8a11-876ea3bc4d25 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:26.125491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ece37f-5541-4ae8-8a11-876ea3bc4d25 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:26.220151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 26/50] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Oct 19 02:11:25 2022] POST /identity/v3/projects => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:28.355006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d5a52-5717-41a7-8d2a-394f5158fa0b None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:28.355485 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d5a52-5717-41a7-8d2a-394f5158fa0b None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:28.355485 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d5a52-5717-41a7-8d2a-394f5158fa0b None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:28.356821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 25/51] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:28.362071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbab968-4659-4695-9a65-ce9dc1ec4480 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:28.362466 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbab968-4659-4695-9a65-ce9dc1ec4480 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:28.362625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbab968-4659-4695-9a65-ce9dc1ec4480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:28.380939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-5bbab968-4659-4695-9a65-ce9dc1ec4480 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:28.382142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5bbab968-4659-4695-9a65-ce9dc1ec4480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:28.423118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:28.446497 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 27/52] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:28.718466 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d5a52-5717-41a7-8d2a-394f5158fa0b None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:28.818785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d5a52-5717-41a7-8d2a-394f5158fa0b None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:28.850367 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d6a40-ec99-47e1-8eb9-ec736d6da7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:28.850990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6a40-ec99-47e1-8eb9-ec736d6da7db None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:28.851273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6a40-ec99-47e1-8eb9-ec736d6da7db None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:28.851548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6a40-ec99-47e1-8eb9-ec736d6da7db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:28.852660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d6a40-ec99-47e1-8eb9-ec736d6da7db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:28.854114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d6a40-ec99-47e1-8eb9-ec736d6da7db None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:28.855656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 26/53] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:28.861646 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbab968-4659-4695-9a65-ce9dc1ec4480 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:28.955644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbab968-4659-4695-9a65-ce9dc1ec4480 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:28.988877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c27b7c-79e8-4644-acd8-c25147511225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:28.989434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c27b7c-79e8-4644-acd8-c25147511225 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:28.989713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c27b7c-79e8-4644-acd8-c25147511225 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:28.989982 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c27b7c-79e8-4644-acd8-c25147511225 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:28.990665 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c27b7c-79e8-4644-acd8-c25147511225 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:28.992093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c27b7c-79e8-4644-acd8-c25147511225 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:29.116969 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 28/54] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:11:28 2022] POST /identity/v3/users => generated 275 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:32.293611 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef15d11-d2b0-4d23-ba9d-26183a5c71f5 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:32.294650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef15d11-d2b0-4d23-ba9d-26183a5c71f5 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:32.295669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef15d11-d2b0-4d23-ba9d-26183a5c71f5 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:32.297222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 27/55] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:32.302614 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a5061-6e0e-444b-a6fd-5e8978dd0a4d None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:32.303427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a5061-6e0e-444b-a6fd-5e8978dd0a4d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:32.303938 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a5061-6e0e-444b-a6fd-5e8978dd0a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:32.330966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-065a5061-6e0e-444b-a6fd-5e8978dd0a4d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:32.332444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-065a5061-6e0e-444b-a6fd-5e8978dd0a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:32.366474 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:32.421160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 29/56] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:32.534400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e61447-12ac-4390-a7bc-d19eae38405f None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:32.535016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e61447-12ac-4390-a7bc-d19eae38405f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:32.535508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e61447-12ac-4390-a7bc-d19eae38405f None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:32.537034 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 28/57] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:32.542239 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f155106-7870-42b3-bcb6-eaaefe0789e4 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:32.542767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f155106-7870-42b3-bcb6-eaaefe0789e4 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:32.543310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f155106-7870-42b3-bcb6-eaaefe0789e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:32.568187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-5f155106-7870-42b3-bcb6-eaaefe0789e4 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:32.569761 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5f155106-7870-42b3-bcb6-eaaefe0789e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:32.661720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 30/58] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:32.710460 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec97496-2e7c-48f8-9e6f-57fc2ffcfe76 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:32.711438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec97496-2e7c-48f8-9e6f-57fc2ffcfe76 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:32.711907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec97496-2e7c-48f8-9e6f-57fc2ffcfe76 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:32.713438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 29/59] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:32.719398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d306d-70cf-41dd-bc8f-00674e7ecfe2 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:32.719953 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d306d-70cf-41dd-bc8f-00674e7ecfe2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:32.720423 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d306d-70cf-41dd-bc8f-00674e7ecfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:32.739221 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-072d306d-70cf-41dd-bc8f-00674e7ecfe2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:32.740777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-072d306d-70cf-41dd-bc8f-00674e7ecfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:32.777246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec97496-2e7c-48f8-9e6f-57fc2ffcfe76 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:32.840327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 31/60] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:32.870422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:32.883916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87e539-87d6-4de7-af6b-9e3601dc5a73 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:32.884402 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87e539-87d6-4de7-af6b-9e3601dc5a73 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:32.884834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87e539-87d6-4de7-af6b-9e3601dc5a73 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:32.886349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 32/61] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:11:32.896716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e53fe-8eab-42b5-bfae-eaa9807e40b9 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:32.897006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e53fe-8eab-42b5-bfae-eaa9807e40b9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:32.897283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e53fe-8eab-42b5-bfae-eaa9807e40b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:32.930740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-525e53fe-8eab-42b5-bfae-eaa9807e40b9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:32.931855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-525e53fe-8eab-42b5-bfae-eaa9807e40b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:32.941452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec97496-2e7c-48f8-9e6f-57fc2ffcfe76 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:32.973537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:32.988055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401cffd-9f1d-4643-838d-15385d88196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:32.988753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401cffd-9f1d-4643-838d-15385d88196c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:32.991228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401cffd-9f1d-4643-838d-15385d88196c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:32.991228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401cffd-9f1d-4643-838d-15385d88196c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.009889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401cffd-9f1d-4643-838d-15385d88196c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-5401cffd-9f1d-4643-838d-15385d88196c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:33.016532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:33.017384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:33.018148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:33.020429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 30/62] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:11:33.021443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401cffd-9f1d-4643-838d-15385d88196c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.041485 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 33/63] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:33.042518 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e53fe-8eab-42b5-bfae-eaa9807e40b9 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.142163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401cffd-9f1d-4643-838d-15385d88196c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:33.156085 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e53fe-8eab-42b5-bfae-eaa9807e40b9 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:33.167569 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:33.189267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254760a6-96de-4d09-a53c-f57c730c3adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:33.189267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254760a6-96de-4d09-a53c-f57c730c3adc None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.189267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254760a6-96de-4d09-a53c-f57c730c3adc None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.189267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254760a6-96de-4d09-a53c-f57c730c3adc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.193795 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d3f6b-3c3a-4ca1-a198-43f7911f185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:33.194650 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3f6b-3c3a-4ca1-a198-43f7911f185c None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.195141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3f6b-3c3a-4ca1-a198-43f7911f185c None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.195620 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3f6b-3c3a-4ca1-a198-43f7911f185c None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.196557 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d3f6b-3c3a-4ca1-a198-43f7911f185c None admin] RBAC: Adding query filter params (name=member) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:33.197064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d3f6b-3c3a-4ca1-a198-43f7911f185c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:33.197979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254760a6-96de-4d09-a53c-f57c730c3adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:33.198937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d3f6b-3c3a-4ca1-a198-43f7911f185c None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-254760a6-96de-4d09-a53c-f57c730c3adc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:33.203824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:33.204999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:33.206028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 31/64] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:33.206671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c707e0-8681-45ad-865a-818b7c4dc410 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.206671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c707e0-8681-45ad-865a-818b7c4dc410 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.207071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c707e0-8681-45ad-865a-818b7c4dc410 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.212305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 34/65] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:33.212811 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 32/66] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:33.213504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c707e0-8681-45ad-865a-818b7c4dc410 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.213602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d3f6b-3c3a-4ca1-a198-43f7911f185c None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.229403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:33.318175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d3f6b-3c3a-4ca1-a198-43f7911f185c None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:33.327910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c707e0-8681-45ad-865a-818b7c4dc410 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:33.351351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744051d8-eeae-4b77-ac50-1c2bcf075df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:33.351970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744051d8-eeae-4b77-ac50-1c2bcf075df1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.352251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744051d8-eeae-4b77-ac50-1c2bcf075df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.352534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744051d8-eeae-4b77-ac50-1c2bcf075df1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.363004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744051d8-eeae-4b77-ac50-1c2bcf075df1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:33.366379 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535b9d7-720f-4459-ac5a-60782289a1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:33.366997 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535b9d7-720f-4459-ac5a-60782289a1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.367281 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535b9d7-720f-4459-ac5a-60782289a1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.367556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535b9d7-720f-4459-ac5a-60782289a1b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-744051d8-eeae-4b77-ac50-1c2bcf075df1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:33.367820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:33.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:33.369563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:33.370143 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535b9d7-720f-4459-ac5a-60782289a1b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:33.370143 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535b9d7-720f-4459-ac5a-60782289a1b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:33.370278 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 35/67] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:33.370338 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535b9d7-720f-4459-ac5a-60782289a1b1 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:33.370799 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fb5ca-7c9d-4f0a-8219-eb8250579fc6 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.370799 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fb5ca-7c9d-4f0a-8219-eb8250579fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.371043 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fb5ca-7c9d-4f0a-8219-eb8250579fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.383723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 33/68] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:33.384531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535b9d7-720f-4459-ac5a-60782289a1b1 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.388062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-3f0fb5ca-7c9d-4f0a-8219-eb8250579fc6 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:33.389293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3f0fb5ca-7c9d-4f0a-8219-eb8250579fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:33.455595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 36/69] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:33.456386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fb5ca-7c9d-4f0a-8219-eb8250579fc6 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.471947 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:33.530918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:33.575341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fb5ca-7c9d-4f0a-8219-eb8250579fc6 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:33.593056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535b9d7-720f-4459-ac5a-60782289a1b1 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:33.612686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71628f-314d-48c6-90bb-cbf26586e95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:33.613323 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71628f-314d-48c6-90bb-cbf26586e95e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.613605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71628f-314d-48c6-90bb-cbf26586e95e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.613876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71628f-314d-48c6-90bb-cbf26586e95e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.624519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71628f-314d-48c6-90bb-cbf26586e95e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:33.628167 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feac66c-c2f5-4a54-bebf-fc1737c82325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:33.628999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feac66c-c2f5-4a54-bebf-fc1737c82325 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-9b71628f-314d-48c6-90bb-cbf26586e95e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:33.629334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:33.630210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 37/70] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:33.635858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71628f-314d-48c6-90bb-cbf26586e95e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.636651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feac66c-c2f5-4a54-bebf-fc1737c82325 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.636651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feac66c-c2f5-4a54-bebf-fc1737c82325 None admin] PATH_INFO: `/v3/users/98f9702915254d03a5ddc76409a170e3` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.655033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feac66c-c2f5-4a54-bebf-fc1737c82325 None admin] RBAC: Authorizing `identity:get_user(user_id=98f9702915254d03a5ddc76409a170e3)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:33.656506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feac66c-c2f5-4a54-bebf-fc1737c82325 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:33.662432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 34/71] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/users/98f9702915254d03a5ddc76409a170e3 => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:33.662432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feac66c-c2f5-4a54-bebf-fc1737c82325 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.702597 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:33.743767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71628f-314d-48c6-90bb-cbf26586e95e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:33.762150 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:33.766385 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feac66c-c2f5-4a54-bebf-fc1737c82325 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:33.777410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d38e7-fb30-4960-86aa-79d96567297e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:33.778014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d38e7-fb30-4960-86aa-79d96567297e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.778327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d38e7-fb30-4960-86aa-79d96567297e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.778611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d38e7-fb30-4960-86aa-79d96567297e None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.779273 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d38e7-fb30-4960-86aa-79d96567297e None admin] RBAC: Adding query filter params (name=member) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:33.779580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d38e7-fb30-4960-86aa-79d96567297e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:33.780979 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d38e7-fb30-4960-86aa-79d96567297e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:33.788803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 38/72] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:33.789592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d38e7-fb30-4960-86aa-79d96567297e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.799813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60986899-cf8c-4336-80c6-7f158bca3af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:33.800443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60986899-cf8c-4336-80c6-7f158bca3af8 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.801084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60986899-cf8c-4336-80c6-7f158bca3af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.801365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60986899-cf8c-4336-80c6-7f158bca3af8 None admin] PATH_INFO: `/v3/users/88e388dc7d10482d8944dbc5c7039f97` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.802554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60986899-cf8c-4336-80c6-7f158bca3af8 None admin] RBAC: Authorizing `identity:get_user(user_id=88e388dc7d10482d8944dbc5c7039f97)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:33.803969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60986899-cf8c-4336-80c6-7f158bca3af8 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:33.805516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 35/73] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/users/88e388dc7d10482d8944dbc5c7039f97 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:33.806302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60986899-cf8c-4336-80c6-7f158bca3af8 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.889104 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d38e7-fb30-4960-86aa-79d96567297e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:33.902510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60986899-cf8c-4336-80c6-7f158bca3af8 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:33.913953 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:33.922397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262183be-32de-4ea4-a080-37db8a535d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:33.923086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262183be-32de-4ea4-a080-37db8a535d72 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.923371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262183be-32de-4ea4-a080-37db8a535d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.923645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262183be-32de-4ea4-a080-37db8a535d72 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.924323 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262183be-32de-4ea4-a080-37db8a535d72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:33.924623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262183be-32de-4ea4-a080-37db8a535d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:33.925992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262183be-32de-4ea4-a080-37db8a535d72 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:33.934198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 39/74] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:33.934997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262183be-32de-4ea4-a080-37db8a535d72 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.935581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dedba-b44d-4acb-ba6b-578d4437f053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:33.936214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dedba-b44d-4acb-ba6b-578d4437f053 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:33.936495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dedba-b44d-4acb-ba6b-578d4437f053 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:33.936767 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727dedba-b44d-4acb-ba6b-578d4437f053 None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:33.938118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727dedba-b44d-4acb-ba6b-578d4437f053 None admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:33.940347 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727dedba-b44d-4acb-ba6b-578d4437f053 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:33.941974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 36/75] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:33.942752 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dedba-b44d-4acb-ba6b-578d4437f053 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:33.976851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:34.034374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262183be-32de-4ea4-a080-37db8a535d72 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:34.040732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727dedba-b44d-4acb-ba6b-578d4437f053 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:34.067262 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de17f8f-8fc0-487a-baca-2294e94cd7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:34.067884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de17f8f-8fc0-487a-baca-2294e94cd7ef None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:34.068172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de17f8f-8fc0-487a-baca-2294e94cd7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:34.068448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de17f8f-8fc0-487a-baca-2294e94cd7ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:34.069610 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de17f8f-8fc0-487a-baca-2294e94cd7ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:34.071093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de17f8f-8fc0-487a-baca-2294e94cd7ef None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:34.072665 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 40/76] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:34.073432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de17f8f-8fc0-487a-baca-2294e94cd7ef None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:34.078542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a92dc-1a3b-4944-8106-6f9deec25dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:34.079180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a92dc-1a3b-4944-8106-6f9deec25dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:34.079465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a92dc-1a3b-4944-8106-6f9deec25dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:34.079741 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a92dc-1a3b-4944-8106-6f9deec25dd4 None admin] PATH_INFO: `/v3/users/98f9702915254d03a5ddc76409a170e3` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:34.080873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a92dc-1a3b-4944-8106-6f9deec25dd4 None admin] RBAC: Authorizing `identity:get_user(user_id=98f9702915254d03a5ddc76409a170e3)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:34.082323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a92dc-1a3b-4944-8106-6f9deec25dd4 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:34.084052 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 37/77] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:33 2022] GET /identity/v3/users/98f9702915254d03a5ddc76409a170e3 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:34.084612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a92dc-1a3b-4944-8106-6f9deec25dd4 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:34.131955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:34.174794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de17f8f-8fc0-487a-baca-2294e94cd7ef None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:34.185201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a92dc-1a3b-4944-8106-6f9deec25dd4 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:34.187239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:34.209485 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168bef3-77d7-4ceb-9de6-439acadb9820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:34.210158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bef3-77d7-4ceb-9de6-439acadb9820 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:34.210438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bef3-77d7-4ceb-9de6-439acadb9820 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:34.210962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168bef3-77d7-4ceb-9de6-439acadb9820 None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:34.212079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168bef3-77d7-4ceb-9de6-439acadb9820 None admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:34.213567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168bef3-77d7-4ceb-9de6-439acadb9820 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:34.215215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 41/78] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:34 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:34.215950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168bef3-77d7-4ceb-9de6-439acadb9820 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:34.221266 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672529c7-4f70-4284-9954-79f0ca427011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:34.221910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672529c7-4f70-4284-9954-79f0ca427011 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:34.222225 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672529c7-4f70-4284-9954-79f0ca427011 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:34.222504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672529c7-4f70-4284-9954-79f0ca427011 None admin] PATH_INFO: `/v3/users/98f9702915254d03a5ddc76409a170e3` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:34.223648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672529c7-4f70-4284-9954-79f0ca427011 None admin] RBAC: Authorizing `identity:get_user(user_id=98f9702915254d03a5ddc76409a170e3)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:34.225519 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672529c7-4f70-4284-9954-79f0ca427011 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:34.232068 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 38/79] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:34 2022] GET /identity/v3/users/98f9702915254d03a5ddc76409a170e3 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:34.232068 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672529c7-4f70-4284-9954-79f0ca427011 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:34.311778 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168bef3-77d7-4ceb-9de6-439acadb9820 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:34.325302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672529c7-4f70-4284-9954-79f0ca427011 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:34.333563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:34.344476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1c45e-d7b7-460b-a4f8-ab5159d575f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:34.345128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1c45e-d7b7-460b-a4f8-ab5159d575f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:34.347458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1c45e-d7b7-460b-a4f8-ab5159d575f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:34.347458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f1c45e-d7b7-460b-a4f8-ab5159d575f6 None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf/users/98f9702915254d03a5ddc76409a170e3/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:34.347768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1c45e-d7b7-460b-a4f8-ab5159d575f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=dce320a8ccf64f479c1dae1117865baf, user_id=98f9702915254d03a5ddc76409a170e3, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:34.349216 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f1c45e-d7b7-460b-a4f8-ab5159d575f6 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:34.359051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf36f18-453f-403f-a1e8-c0c23ec8eaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:34.359659 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf36f18-453f-403f-a1e8-c0c23ec8eaab None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:34.359936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf36f18-453f-403f-a1e8-c0c23ec8eaab None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:34.360204 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf36f18-453f-403f-a1e8-c0c23ec8eaab None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:34.360872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf36f18-453f-403f-a1e8-c0c23ec8eaab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:34.362290 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf36f18-453f-403f-a1e8-c0c23ec8eaab None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:34.419014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 42/80] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:11:34 2022] PUT /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf/users/98f9702915254d03a5ddc76409a170e3/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:11:34.419826 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1c45e-d7b7-460b-a4f8-ab5159d575f6 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:34.454538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 39/81] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:11:34 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:34.455375 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf36f18-453f-403f-a1e8-c0c23ec8eaab None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:34.494534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:34.554744 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f1c45e-d7b7-460b-a4f8-ab5159d575f6 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:34.589960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf36f18-453f-403f-a1e8-c0c23ec8eaab None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:34.595318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06708c0e-9636-400c-bca0-24268a5d6db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:34.596361 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06708c0e-9636-400c-bca0-24268a5d6db4 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:34.596873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06708c0e-9636-400c-bca0-24268a5d6db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:34.597354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06708c0e-9636-400c-bca0-24268a5d6db4 None admin] PATH_INFO: `/v3/projects/b6fff516c6ad43ba88f4cb7b264e31ab` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:34.598983 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06708c0e-9636-400c-bca0-24268a5d6db4 None admin] RBAC: Authorizing `identity:get_project(project_id=b6fff516c6ad43ba88f4cb7b264e31ab)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:34.600803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06708c0e-9636-400c-bca0-24268a5d6db4 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:34.602905 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 43/82] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:34 2022] GET /identity/v3/projects/b6fff516c6ad43ba88f4cb7b264e31ab => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:34.604039 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06708c0e-9636-400c-bca0-24268a5d6db4 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:34.631078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a2196-e58e-4e1e-a545-c43f8654e9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:34.631771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a2196-e58e-4e1e-a545-c43f8654e9d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:34.632051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a2196-e58e-4e1e-a545-c43f8654e9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:34.632386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a2196-e58e-4e1e-a545-c43f8654e9d1 None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf/users/88e388dc7d10482d8944dbc5c7039f97/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:34.634674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a2196-e58e-4e1e-a545-c43f8654e9d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=dce320a8ccf64f479c1dae1117865baf, user_id=88e388dc7d10482d8944dbc5c7039f97, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:34.636108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a2196-e58e-4e1e-a545-c43f8654e9d1 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:34.666890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:34.716532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06708c0e-9636-400c-bca0-24268a5d6db4 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:34.755869 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578a42a-499b-4298-94dd-248528117598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:34.756512 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578a42a-499b-4298-94dd-248528117598 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:34.756855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578a42a-499b-4298-94dd-248528117598 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:34.757122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578a42a-499b-4298-94dd-248528117598 None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:34.758366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578a42a-499b-4298-94dd-248528117598 None admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:34.759883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578a42a-499b-4298-94dd-248528117598 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:34.763058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 44/83] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:34 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:34.763058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578a42a-499b-4298-94dd-248528117598 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:34.803674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 40/84] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:11:34 2022] PUT /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf/users/88e388dc7d10482d8944dbc5c7039f97/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:11:34.804424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a2196-e58e-4e1e-a545-c43f8654e9d1 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:34.892126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578a42a-499b-4298-94dd-248528117598 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:34.916089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:34.919812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:34.934613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fc50c-b5c6-461a-89c2-b7d3c830482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:34.936282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fc50c-b5c6-461a-89c2-b7d3c830482c None admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:34.936766 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fc50c-b5c6-461a-89c2-b7d3c830482c None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:34.937283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fc50c-b5c6-461a-89c2-b7d3c830482c None admin] PATH_INFO: `/v3/projects/b6fff516c6ad43ba88f4cb7b264e31ab/users/98f9702915254d03a5ddc76409a170e3/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:34.939206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a2196-e58e-4e1e-a545-c43f8654e9d1 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:34.939875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69fc50c-b5c6-461a-89c2-b7d3c830482c None admin] RBAC: Authorizing `identity:create_grant(project_id=b6fff516c6ad43ba88f4cb7b264e31ab, user_id=98f9702915254d03a5ddc76409a170e3, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:34.941594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69fc50c-b5c6-461a-89c2-b7d3c830482c None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:34.981474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c571bd5-6bf7-43c9-978f-b5c66f6bfcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:34.982222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c571bd5-6bf7-43c9-978f-b5c66f6bfcac None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:34.982624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c571bd5-6bf7-43c9-978f-b5c66f6bfcac None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:34.982903 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c571bd5-6bf7-43c9-978f-b5c66f6bfcac None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf/users/98f9702915254d03a5ddc76409a170e3/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:34.985397 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c571bd5-6bf7-43c9-978f-b5c66f6bfcac None admin] RBAC: Authorizing `identity:create_grant(project_id=dce320a8ccf64f479c1dae1117865baf, user_id=98f9702915254d03a5ddc76409a170e3, role_id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:34.987132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c571bd5-6bf7-43c9-978f-b5c66f6bfcac None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:35.022294 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 45/85] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:11:34 2022] PUT /identity/v3/projects/b6fff516c6ad43ba88f4cb7b264e31ab/users/98f9702915254d03a5ddc76409a170e3/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:11:35.084177 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 41/86] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:11:34 2022] PUT /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf/users/98f9702915254d03a5ddc76409a170e3/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:11:37.080077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605df58-21c8-45c2-99db-339c4451e3e0 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:37.098926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605df58-21c8-45c2-99db-339c4451e3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:37.098926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605df58-21c8-45c2-99db-339c4451e3e0 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:37.098926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 46/87] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:37.099196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9edec9-f1a4-4bba-b43d-c0e96b1c94eb None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:37.099196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9edec9-f1a4-4bba-b43d-c0e96b1c94eb None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:37.099196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9edec9-f1a4-4bba-b43d-c0e96b1c94eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:37.115092 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-bc9edec9-f1a4-4bba-b43d-c0e96b1c94eb None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:37.116302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-bc9edec9-f1a4-4bba-b43d-c0e96b1c94eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:37.200729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 42/88] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:37.526870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a43ca-74f4-4a73-9a08-4cf168a3e460 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:37.527575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a43ca-74f4-4a73-9a08-4cf168a3e460 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:37.528074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a43ca-74f4-4a73-9a08-4cf168a3e460 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:37.530205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 47/89] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:37.535418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccecded5-299f-473f-bc22-c2da30671002 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:37.535703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccecded5-299f-473f-bc22-c2da30671002 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:37.535976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccecded5-299f-473f-bc22-c2da30671002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:37.559181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ccecded5-299f-473f-bc22-c2da30671002 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:37.560389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ccecded5-299f-473f-bc22-c2da30671002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:37.644485 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 43/90] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:37.751091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a43ca-74f4-4a73-9a08-4cf168a3e460 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:37.828334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd21f78-92ff-458b-b2a9-7ea186814a3f None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:37.828631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd21f78-92ff-458b-b2a9-7ea186814a3f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:37.828884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd21f78-92ff-458b-b2a9-7ea186814a3f None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:37.830448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 44/91] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:11:37.837002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda49f7-f2a0-4771-b059-a07f053d68b2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:37.837285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda49f7-f2a0-4771-b059-a07f053d68b2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:37.837565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda49f7-f2a0-4771-b059-a07f053d68b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:37.858413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:37.861302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-bcda49f7-f2a0-4771-b059-a07f053d68b2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:37.862766 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-bcda49f7-f2a0-4771-b059-a07f053d68b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:37.921039 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a43ca-74f4-4a73-9a08-4cf168a3e460 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:37.939969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 45/92] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:11:37.941463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168128c1-a8f4-46db-b765-c65092c5c7dc None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:37.941753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168128c1-a8f4-46db-b765-c65092c5c7dc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:37.942203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168128c1-a8f4-46db-b765-c65092c5c7dc None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:37.943444 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 46/93] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:37.948680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9197bb-348b-4752-a958-3796d3514c76 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:37.949231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9197bb-348b-4752-a958-3796d3514c76 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:37.949509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9197bb-348b-4752-a958-3796d3514c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:37.972287 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ac9197bb-348b-4752-a958-3796d3514c76 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:37.973539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ac9197bb-348b-4752-a958-3796d3514c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:38.010395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6c9fa-8b41-433c-a252-43a2ac7b7ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:38.012017 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6c9fa-8b41-433c-a252-43a2ac7b7ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:38.012558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6c9fa-8b41-433c-a252-43a2ac7b7ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:38.012827 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6c9fa-8b41-433c-a252-43a2ac7b7ed1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:38.051553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6c9fa-8b41-433c-a252-43a2ac7b7ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:38.063185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 47/94] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-8df6c9fa-8b41-433c-a252-43a2ac7b7ed1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:38.066797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:38.067904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:38.068725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:38.069810 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 48/95] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:38.075043 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9197bb-348b-4752-a958-3796d3514c76 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:38.166603 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6c9fa-8b41-433c-a252-43a2ac7b7ed1 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:38.190387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:38.217000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:38.277599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9197bb-348b-4752-a958-3796d3514c76 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:38.324723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d0527-1d3b-4aa2-8a42-69db73394f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:38.327504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6c9fa-8b41-433c-a252-43a2ac7b7ed1 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:38.328370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d0527-1d3b-4aa2-8a42-69db73394f0d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:38.328936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d0527-1d3b-4aa2-8a42-69db73394f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:38.329424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d0527-1d3b-4aa2-8a42-69db73394f0d None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:38.330328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d0527-1d3b-4aa2-8a42-69db73394f0d None admin] RBAC: Adding query filter params (name=member) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:38.332180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d0527-1d3b-4aa2-8a42-69db73394f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:38.333792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d0527-1d3b-4aa2-8a42-69db73394f0d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:38.347500 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 48/96] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:38.347500 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d0527-1d3b-4aa2-8a42-69db73394f0d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:38.397750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ac003-ad08-444c-a7e8-fe90888f1bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:38.398774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ac003-ad08-444c-a7e8-fe90888f1bac None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:38.399260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ac003-ad08-444c-a7e8-fe90888f1bac None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:38.399829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ac003-ad08-444c-a7e8-fe90888f1bac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:38.425193 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ac003-ad08-444c-a7e8-fe90888f1bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-943ac003-ad08-444c-a7e8-fe90888f1bac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:38.430765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:38.431602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:11:38.432439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:38.434801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 49/97] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:38.440589 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ac003-ad08-444c-a7e8-fe90888f1bac None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:38.461688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d0527-1d3b-4aa2-8a42-69db73394f0d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:38.491592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:38.496822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e9ba1-960d-4b02-93cc-f38e8173e382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:38.497684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e9ba1-960d-4b02-93cc-f38e8173e382 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:38.498182 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e9ba1-960d-4b02-93cc-f38e8173e382 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:38.498669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e9ba1-960d-4b02-93cc-f38e8173e382 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:38.499952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e9ba1-960d-4b02-93cc-f38e8173e382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:38.501922 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813e9ba1-960d-4b02-93cc-f38e8173e382 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:38.503679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 49/98] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:38.504724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e9ba1-960d-4b02-93cc-f38e8173e382 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:38.613776 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e9ba1-960d-4b02-93cc-f38e8173e382 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:38.619397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:38.624036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ac003-ad08-444c-a7e8-fe90888f1bac None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:38.656305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de975c8-e2d2-429b-9223-487d24eb303a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:38.657193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de975c8-e2d2-429b-9223-487d24eb303a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:38.657610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de975c8-e2d2-429b-9223-487d24eb303a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:38.658028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de975c8-e2d2-429b-9223-487d24eb303a None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:38.658929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de975c8-e2d2-429b-9223-487d24eb303a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:38.659425 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de975c8-e2d2-429b-9223-487d24eb303a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:38.661165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de975c8-e2d2-429b-9223-487d24eb303a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:38.668525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983cfcc-08a5-441d-b38d-3a62355101c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:38.669182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983cfcc-08a5-441d-b38d-3a62355101c7 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:38.669538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983cfcc-08a5-441d-b38d-3a62355101c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:38.669919 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983cfcc-08a5-441d-b38d-3a62355101c7 None admin] PATH_INFO: `/v3/users/98f9702915254d03a5ddc76409a170e3` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:38.671239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 50/99] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:38.671352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983cfcc-08a5-441d-b38d-3a62355101c7 None admin] RBAC: Authorizing `identity:get_user(user_id=98f9702915254d03a5ddc76409a170e3)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:38.672327 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de975c8-e2d2-429b-9223-487d24eb303a None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:38.673199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983cfcc-08a5-441d-b38d-3a62355101c7 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:38.674903 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 50/100] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:38 2022] GET /identity/v3/users/98f9702915254d03a5ddc76409a170e3 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:38.676008 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983cfcc-08a5-441d-b38d-3a62355101c7 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:38.732860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:38.774988 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de975c8-e2d2-429b-9223-487d24eb303a None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:38.784242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983cfcc-08a5-441d-b38d-3a62355101c7 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:38.809059 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f16f4-dfb9-462c-a762-65053145dcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:38.809668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f16f4-dfb9-462c-a762-65053145dcad None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:38.809945 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f16f4-dfb9-462c-a762-65053145dcad None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:38.810307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f16f4-dfb9-462c-a762-65053145dcad None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:38.811497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f16f4-dfb9-462c-a762-65053145dcad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:38.812937 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f16f4-dfb9-462c-a762-65053145dcad None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:38.816761 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2605acf-2d16-449a-8f4a-23194ea20980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:38.817390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2605acf-2d16-449a-8f4a-23194ea20980 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:38.817664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2605acf-2d16-449a-8f4a-23194ea20980 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:38.817931 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2605acf-2d16-449a-8f4a-23194ea20980 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:38.827701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2605acf-2d16-449a-8f4a-23194ea20980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-d2605acf-2d16-449a-8f4a-23194ea20980 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:38.832428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:38.833279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 51/101] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:11:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:38.834289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2605acf-2d16-449a-8f4a-23194ea20980 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:38.861999 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:38.927151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 51/102] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:11:38 2022] POST /identity/v3/projects => generated 302 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:38.927151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f16f4-dfb9-462c-a762-65053145dcad None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:38.940369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2605acf-2d16-449a-8f4a-23194ea20980 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:38.979328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d0e9b-c734-420a-a41c-a1ef81e4870a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:38.980074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d0e9b-c734-420a-a41c-a1ef81e4870a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:38.980362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d0e9b-c734-420a-a41c-a1ef81e4870a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:38.980635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d0e9b-c734-420a-a41c-a1ef81e4870a None admin] PATH_INFO: `/v3/users/19fa4560197a46b5bff8ec65cd3f6d84` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:38.996274 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d0e9b-c734-420a-a41c-a1ef81e4870a None admin] RBAC: Authorizing `identity:get_user(user_id=19fa4560197a46b5bff8ec65cd3f6d84)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:38.997779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d0e9b-c734-420a-a41c-a1ef81e4870a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:39.000236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 52/103] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:38 2022] GET /identity/v3/users/19fa4560197a46b5bff8ec65cd3f6d84 => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:39.006001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d846-ea2d-4669-9c05-d1c094e95029 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:39.006316 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d846-ea2d-4669-9c05-d1c094e95029 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:39.006579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d846-ea2d-4669-9c05-d1c094e95029 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:39.012132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 53/104] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:39.012132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d7b1c-4048-498a-b6fe-f2f796c6db88 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:39.012132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d7b1c-4048-498a-b6fe-f2f796c6db88 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:39.012132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d7b1c-4048-498a-b6fe-f2f796c6db88 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:39.012132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 54/105] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:39.012132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d7b1c-4048-498a-b6fe-f2f796c6db88 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:39.040138 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f16f4-dfb9-462c-a762-65053145dcad None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:39.053334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:39.077691 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcdf04-649c-4616-b4ba-9cd69bfb7fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:39.078360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcdf04-649c-4616-b4ba-9cd69bfb7fca None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:39.078637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcdf04-649c-4616-b4ba-9cd69bfb7fca None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:39.078910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcdf04-649c-4616-b4ba-9cd69bfb7fca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:39.117430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:39.135889 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d7b1c-4048-498a-b6fe-f2f796c6db88 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:39.158507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcdf04-649c-4616-b4ba-9cd69bfb7fca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:39.159961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcdf04-649c-4616-b4ba-9cd69bfb7fca None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:39.173620 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6a68b-1ab5-4ced-bae1-ccaa6d1e1814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:39.174377 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6a68b-1ab5-4ced-bae1-ccaa6d1e1814 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:39.174692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6a68b-1ab5-4ced-bae1-ccaa6d1e1814 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:39.175036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6a68b-1ab5-4ced-bae1-ccaa6d1e1814 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:39.175707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6a68b-1ab5-4ced-bae1-ccaa6d1e1814 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:39.176012 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6a68b-1ab5-4ced-bae1-ccaa6d1e1814 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:39.177439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6a68b-1ab5-4ced-bae1-ccaa6d1e1814 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:39.186189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 55/106] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:39.191173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6a68b-1ab5-4ced-bae1-ccaa6d1e1814 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:39.271862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 52/107] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:11:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:11:39.271862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea325fd4-3907-4281-aa2f-2e296fb3d0e5 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:39.271862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea325fd4-3907-4281-aa2f-2e296fb3d0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:39.271862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea325fd4-3907-4281-aa2f-2e296fb3d0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:39.288832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ea325fd4-3907-4281-aa2f-2e296fb3d0e5 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:39.290028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ea325fd4-3907-4281-aa2f-2e296fb3d0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:39.293585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:39.299235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6a68b-1ab5-4ced-bae1-ccaa6d1e1814 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:39.341701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf9475-3545-45a4-b1fa-10d319fb42e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:39.342495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf9475-3545-45a4-b1fa-10d319fb42e8 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:39.342775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf9475-3545-45a4-b1fa-10d319fb42e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:39.343098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf9475-3545-45a4-b1fa-10d319fb42e8 None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:39.344232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf9475-3545-45a4-b1fa-10d319fb42e8 None admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:39.345696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf9475-3545-45a4-b1fa-10d319fb42e8 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:39.351898 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 56/108] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:39 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:39.351898 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76f8af-6906-4580-ad42-00ee0b660197 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:39.351898 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76f8af-6906-4580-ad42-00ee0b660197 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:39.351898 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76f8af-6906-4580-ad42-00ee0b660197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:39.365405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 53/109] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:39.366102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea325fd4-3907-4281-aa2f-2e296fb3d0e5 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:39.369854 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-ca76f8af-6906-4580-ad42-00ee0b660197 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:39.371268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ca76f8af-6906-4580-ad42-00ee0b660197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:39.386942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:39.453897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 57/110] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:39.455076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76f8af-6906-4580-ad42-00ee0b660197 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:39.491180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea325fd4-3907-4281-aa2f-2e296fb3d0e5 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:39.557897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b77837-d579-477d-aa42-b19f284fcaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:39.558890 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b77837-d579-477d-aa42-b19f284fcaed None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:39.559384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b77837-d579-477d-aa42-b19f284fcaed None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:39.559896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b77837-d579-477d-aa42-b19f284fcaed None admin] PATH_INFO: `/v3/users/88e388dc7d10482d8944dbc5c7039f97` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:39.561740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b77837-d579-477d-aa42-b19f284fcaed None admin] RBAC: Authorizing `identity:get_user(user_id=88e388dc7d10482d8944dbc5c7039f97)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:39.563498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b77837-d579-477d-aa42-b19f284fcaed None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:39.565961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 54/111] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:39 2022] GET /identity/v3/users/88e388dc7d10482d8944dbc5c7039f97 => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:39.567078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b77837-d579-477d-aa42-b19f284fcaed None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:39.579337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:39.608656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76f8af-6906-4580-ad42-00ee0b660197 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:39.666822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49f0b9-d831-4b67-b7de-17f83786d05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:39.667855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49f0b9-d831-4b67-b7de-17f83786d05e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:39.668138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49f0b9-d831-4b67-b7de-17f83786d05e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:39.668413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49f0b9-d831-4b67-b7de-17f83786d05e None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:39.669800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49f0b9-d831-4b67-b7de-17f83786d05e None admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:39.671406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e49f0b9-d831-4b67-b7de-17f83786d05e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:39.674227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 58/112] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:39 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:39.679172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49f0b9-d831-4b67-b7de-17f83786d05e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:39.697893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:39.721711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b77837-d579-477d-aa42-b19f284fcaed None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:39.764238 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223bfbb-2e1b-4759-912d-7f1e1d77c4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:39.765129 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223bfbb-2e1b-4759-912d-7f1e1d77c4ca None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:39.765653 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223bfbb-2e1b-4759-912d-7f1e1d77c4ca None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:39.766320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a223bfbb-2e1b-4759-912d-7f1e1d77c4ca None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf/users/19fa4560197a46b5bff8ec65cd3f6d84/roles/dda0ba34c8b9442d9e12b1c8ab9a5d6c` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:39.769178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223bfbb-2e1b-4759-912d-7f1e1d77c4ca None admin] RBAC: Authorizing `identity:create_grant(project_id=dce320a8ccf64f479c1dae1117865baf, user_id=19fa4560197a46b5bff8ec65cd3f6d84, role_id=dda0ba34c8b9442d9e12b1c8ab9a5d6c)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:39.770884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a223bfbb-2e1b-4759-912d-7f1e1d77c4ca None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:39.785275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49f0b9-d831-4b67-b7de-17f83786d05e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:39.818344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae31b1-bbe8-4937-a9b2-ffc2406345c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:39.818949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae31b1-bbe8-4937-a9b2-ffc2406345c2 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:39.819292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae31b1-bbe8-4937-a9b2-ffc2406345c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:39.819570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae31b1-bbe8-4937-a9b2-ffc2406345c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:39.832128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 55/113] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:11:39 2022] PUT /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf/users/19fa4560197a46b5bff8ec65cd3f6d84/roles/dda0ba34c8b9442d9e12b1c8ab9a5d6c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:11:39.843693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223bfbb-2e1b-4759-912d-7f1e1d77c4ca None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:39.907937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae31b1-bbe8-4937-a9b2-ffc2406345c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:39.909545 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae31b1-bbe8-4937-a9b2-ffc2406345c2 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:39.928760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:39.945054 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:39.993585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a223bfbb-2e1b-4759-912d-7f1e1d77c4ca None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:40.032530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 59/114] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:11:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:11:40.033527 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae31b1-bbe8-4937-a9b2-ffc2406345c2 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:40.046247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed4097-f59f-433b-bd8c-8eb9369c8bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:40.047434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4097-f59f-433b-bd8c-8eb9369c8bce None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:40.047716 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4097-f59f-433b-bd8c-8eb9369c8bce None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:40.048407 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ed4097-f59f-433b-bd8c-8eb9369c8bce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:40.060184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ed4097-f59f-433b-bd8c-8eb9369c8bce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-a4ed4097-f59f-433b-bd8c-8eb9369c8bce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:40.066613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:40.067459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:40.068192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 56/115] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:11:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:40.068819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed4097-f59f-433b-bd8c-8eb9369c8bce None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:40.159548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae31b1-bbe8-4937-a9b2-ffc2406345c2 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:40.176687 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:40.185089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ed4097-f59f-433b-bd8c-8eb9369c8bce None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:40.202582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cdc68-938f-4f48-8467-c33015905354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:40.203451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cdc68-938f-4f48-8467-c33015905354 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:40.203740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cdc68-938f-4f48-8467-c33015905354 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:40.204018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cdc68-938f-4f48-8467-c33015905354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:40.205200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cdc68-938f-4f48-8467-c33015905354 None admin] RBAC: Adding query filter params (role.id=bb36a823b2ff45d29018aedf19ae7bf0, scope.project.id=dce320a8ccf64f479c1dae1117865baf, user.id=98f9702915254d03a5ddc76409a170e3) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:40.205525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cdc68-938f-4f48-8467-c33015905354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:40.207027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177cdc68-938f-4f48-8467-c33015905354 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:40.215559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 60/116] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:11:40 2022] GET /identity/v3/role_assignments?user.id=98f9702915254d03a5ddc76409a170e3&scope.project.id=dce320a8ccf64f479c1dae1117865baf&role.id=bb36a823b2ff45d29018aedf19ae7bf0 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:40.216394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cdc68-938f-4f48-8467-c33015905354 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:40.221692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:40.226761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8b0e5-1b51-431d-9a12-21e1c1553518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:40.227872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8b0e5-1b51-431d-9a12-21e1c1553518 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:40.228344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8b0e5-1b51-431d-9a12-21e1c1553518 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:40.228852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8b0e5-1b51-431d-9a12-21e1c1553518 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:40.242036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8b0e5-1b51-431d-9a12-21e1c1553518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-bdf8b0e5-1b51-431d-9a12-21e1c1553518 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:40.248950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:40.249806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:40.250579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:40.253010 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 57/117] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:40.259135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8b0e5-1b51-431d-9a12-21e1c1553518 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:40.377403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cdc68-938f-4f48-8467-c33015905354 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:40.439068 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8b0e5-1b51-431d-9a12-21e1c1553518 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:40.447797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92cc9c-7f9f-4295-947e-e07d5bab5d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:40.448663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92cc9c-7f9f-4295-947e-e07d5bab5d8d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:40.449081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92cc9c-7f9f-4295-947e-e07d5bab5d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:40.449493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92cc9c-7f9f-4295-947e-e07d5bab5d8d None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:40.451097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92cc9c-7f9f-4295-947e-e07d5bab5d8d None admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:40.452915 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92cc9c-7f9f-4295-947e-e07d5bab5d8d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:40.455406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 61/118] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:40 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:40.456246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92cc9c-7f9f-4295-947e-e07d5bab5d8d None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:40.481736 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96579bfc-c876-45b1-8398-7bb6a7e771c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:40.482809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96579bfc-c876-45b1-8398-7bb6a7e771c5 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:40.483307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96579bfc-c876-45b1-8398-7bb6a7e771c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:40.483773 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96579bfc-c876-45b1-8398-7bb6a7e771c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:40.484713 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96579bfc-c876-45b1-8398-7bb6a7e771c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:40.485206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96579bfc-c876-45b1-8398-7bb6a7e771c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:40.486889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96579bfc-c876-45b1-8398-7bb6a7e771c5 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:40.496422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 58/119] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:11:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:40.497476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96579bfc-c876-45b1-8398-7bb6a7e771c5 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:40.522124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:40.561165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:40.604124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92cc9c-7f9f-4295-947e-e07d5bab5d8d None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:40.622719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96579bfc-c876-45b1-8398-7bb6a7e771c5 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:40.649186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd11f63-0831-4582-8ba4-8f3130190213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:40.649800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd11f63-0831-4582-8ba4-8f3130190213 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:40.650099 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd11f63-0831-4582-8ba4-8f3130190213 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:40.650377 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd11f63-0831-4582-8ba4-8f3130190213 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:40.651135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd11f63-0831-4582-8ba4-8f3130190213 None admin] RBAC: Adding query filter params (name=member) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:40.651439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd11f63-0831-4582-8ba4-8f3130190213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:40.653227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd11f63-0831-4582-8ba4-8f3130190213 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:40.665205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 62/120] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:40.665873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c668e-4d77-48df-a950-1849a564bf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:40.666145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd11f63-0831-4582-8ba4-8f3130190213 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:40.666581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c668e-4d77-48df-a950-1849a564bf2b None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:40.666867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c668e-4d77-48df-a950-1849a564bf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:40.667144 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c668e-4d77-48df-a950-1849a564bf2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:40.667893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c668e-4d77-48df-a950-1849a564bf2b None admin] RBAC: Adding query filter params (role.id=d5070bb151d343ca8de32abe373f4a73, scope.project.id=dce320a8ccf64f479c1dae1117865baf, user.id=88e388dc7d10482d8944dbc5c7039f97) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:40.668209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c668e-4d77-48df-a950-1849a564bf2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:40.669670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c668e-4d77-48df-a950-1849a564bf2b None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:40.680352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 59/121] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:11:40 2022] GET /identity/v3/role_assignments?user.id=88e388dc7d10482d8944dbc5c7039f97&scope.project.id=dce320a8ccf64f479c1dae1117865baf&role.id=d5070bb151d343ca8de32abe373f4a73 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:40.683143 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c668e-4d77-48df-a950-1849a564bf2b None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:40.802152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c668e-4d77-48df-a950-1849a564bf2b None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:40.829823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:40.844944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd11f63-0831-4582-8ba4-8f3130190213 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:40.860534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:40.884188 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef0eb5-f046-40db-9527-868c0a9b1fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:40.884847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef0eb5-f046-40db-9527-868c0a9b1fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:40.885126 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef0eb5-f046-40db-9527-868c0a9b1fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:40.885398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef0eb5-f046-40db-9527-868c0a9b1fd7 None admin] PATH_INFO: `/v3/users/98f9702915254d03a5ddc76409a170e3` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:40.888122 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef0eb5-f046-40db-9527-868c0a9b1fd7 None admin] RBAC: Authorizing `identity:get_user(user_id=98f9702915254d03a5ddc76409a170e3)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:40.889636 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef0eb5-f046-40db-9527-868c0a9b1fd7 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:40.891380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 60/122] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:40 2022] GET /identity/v3/users/98f9702915254d03a5ddc76409a170e3 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:40.897029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef0eb5-f046-40db-9527-868c0a9b1fd7 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:40.919980 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb4358-5042-443e-aa16-41f169ad6162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:40.921135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb4358-5042-443e-aa16-41f169ad6162 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:40.921414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb4358-5042-443e-aa16-41f169ad6162 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:40.921692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb4358-5042-443e-aa16-41f169ad6162 None admin] PATH_INFO: `/v3/users/98f9702915254d03a5ddc76409a170e3` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:40.923679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb4358-5042-443e-aa16-41f169ad6162 None admin] RBAC: Authorizing `identity:get_user(user_id=98f9702915254d03a5ddc76409a170e3)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:40.925157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb4358-5042-443e-aa16-41f169ad6162 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:40.926849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 63/123] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:40 2022] GET /identity/v3/users/98f9702915254d03a5ddc76409a170e3 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:40.939344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb4358-5042-443e-aa16-41f169ad6162 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:41.042306 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef0eb5-f046-40db-9527-868c0a9b1fd7 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:41.082894 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb4358-5042-443e-aa16-41f169ad6162 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:41.091016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8b5f6-3e78-4278-8c03-5b0985710e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:41.092566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8b5f6-3e78-4278-8c03-5b0985710e21 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:41.092870 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8b5f6-3e78-4278-8c03-5b0985710e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:41.093141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8b5f6-3e78-4278-8c03-5b0985710e21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:41.131736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bea90b-2e79-4a1c-b3b8-f9110a7a94f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:41.132412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bea90b-2e79-4a1c-b3b8-f9110a7a94f1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:41.132697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bea90b-2e79-4a1c-b3b8-f9110a7a94f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:41.133024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bea90b-2e79-4a1c-b3b8-f9110a7a94f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:41.160804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:41.188682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8b5f6-3e78-4278-8c03-5b0985710e21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:41.190707 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8b5f6-3e78-4278-8c03-5b0985710e21 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:41.205219 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:41.236606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bea90b-2e79-4a1c-b3b8-f9110a7a94f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:41.239441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bea90b-2e79-4a1c-b3b8-f9110a7a94f1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:41.325638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 61/124] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:11:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 19 02:11:41.325638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd761d-365c-4e7b-a85a-b7bb8291c6a5 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:41.325638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd761d-365c-4e7b-a85a-b7bb8291c6a5 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:41.325638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd761d-365c-4e7b-a85a-b7bb8291c6a5 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:41.325638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 62/125] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:41.327841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd761d-365c-4e7b-a85a-b7bb8291c6a5 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:41.360075 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 64/126] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:11:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:11:41.361563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00582eb9-35e2-420b-8701-d9d000ef30b9 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:41.361845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00582eb9-35e2-420b-8701-d9d000ef30b9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:41.362152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00582eb9-35e2-420b-8701-d9d000ef30b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:41.380313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-00582eb9-35e2-420b-8701-d9d000ef30b9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:41.381481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-00582eb9-35e2-420b-8701-d9d000ef30b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:41.422503 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:41.441552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd761d-365c-4e7b-a85a-b7bb8291c6a5 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:41.453963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 65/127] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:41.454983 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00582eb9-35e2-420b-8701-d9d000ef30b9 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:41.465900 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:41.486096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d96524-2f81-430b-b45f-b1f5d19004ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:41.486901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d96524-2f81-430b-b45f-b1f5d19004ca None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:41.487422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d96524-2f81-430b-b45f-b1f5d19004ca None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:41.487625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d96524-2f81-430b-b45f-b1f5d19004ca None admin] PATH_INFO: `/v3/projects/b6fff516c6ad43ba88f4cb7b264e31ab` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:41.488847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d96524-2f81-430b-b45f-b1f5d19004ca None admin] RBAC: Authorizing `identity:get_project(project_id=b6fff516c6ad43ba88f4cb7b264e31ab)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:41.490404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d96524-2f81-430b-b45f-b1f5d19004ca None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:41.497101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 63/128] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:41 2022] GET /identity/v3/projects/b6fff516c6ad43ba88f4cb7b264e31ab => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:41.498099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d96524-2f81-430b-b45f-b1f5d19004ca None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:41.566619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00582eb9-35e2-420b-8701-d9d000ef30b9 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:41.598509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d96524-2f81-430b-b45f-b1f5d19004ca None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:41.602970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3df03-7e91-48d7-9cd8-a244f3a1770a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:41.603793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3df03-7e91-48d7-9cd8-a244f3a1770a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:41.604217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3df03-7e91-48d7-9cd8-a244f3a1770a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:41.604642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3df03-7e91-48d7-9cd8-a244f3a1770a None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:41.605911 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3df03-7e91-48d7-9cd8-a244f3a1770a None admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:41.608518 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3df03-7e91-48d7-9cd8-a244f3a1770a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:41.611076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 66/129] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:41 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:41.616472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3df03-7e91-48d7-9cd8-a244f3a1770a None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:41.634651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11886f4-251e-4142-94f6-d3f7bd1d3727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:41.635449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11886f4-251e-4142-94f6-d3f7bd1d3727 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:41.635845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11886f4-251e-4142-94f6-d3f7bd1d3727 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:41.636244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11886f4-251e-4142-94f6-d3f7bd1d3727 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:41.637352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11886f4-251e-4142-94f6-d3f7bd1d3727 None admin] RBAC: Adding query filter params (role.id=bb36a823b2ff45d29018aedf19ae7bf0, scope.project.id=b6fff516c6ad43ba88f4cb7b264e31ab, user.id=98f9702915254d03a5ddc76409a170e3) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:41.637535 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11886f4-251e-4142-94f6-d3f7bd1d3727 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:41.639637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11886f4-251e-4142-94f6-d3f7bd1d3727 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:41.646178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 64/130] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:11:41 2022] GET /identity/v3/role_assignments?user.id=98f9702915254d03a5ddc76409a170e3&scope.project.id=b6fff516c6ad43ba88f4cb7b264e31ab&role.id=bb36a823b2ff45d29018aedf19ae7bf0 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:41.706512 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:41.733948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11886f4-251e-4142-94f6-d3f7bd1d3727 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:41.736010 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3df03-7e91-48d7-9cd8-a244f3a1770a None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:41.758479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:41.783452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384f574-ba3f-4d19-a90b-f2d8ce5cd1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:41.784076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384f574-ba3f-4d19-a90b-f2d8ce5cd1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:41.784359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384f574-ba3f-4d19-a90b-f2d8ce5cd1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:41.784631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384f574-ba3f-4d19-a90b-f2d8ce5cd1c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:41.785895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384f574-ba3f-4d19-a90b-f2d8ce5cd1c1 None admin] RBAC: Adding query filter params (role.id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6, scope.project.id=dce320a8ccf64f479c1dae1117865baf, user.id=98f9702915254d03a5ddc76409a170e3) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:41.786240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384f574-ba3f-4d19-a90b-f2d8ce5cd1c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:41.788407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384f574-ba3f-4d19-a90b-f2d8ce5cd1c1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:41.796835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 67/131] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:11:41 2022] GET /identity/v3/role_assignments?user.id=98f9702915254d03a5ddc76409a170e3&scope.project.id=dce320a8ccf64f479c1dae1117865baf&role.id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:41.856397 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11886f4-251e-4142-94f6-d3f7bd1d3727 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:41.898887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800af5f6-f256-414c-be42-e846cf41da2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:41.899451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af5f6-f256-414c-be42-e846cf41da2c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:41.899731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af5f6-f256-414c-be42-e846cf41da2c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:41.900002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800af5f6-f256-414c-be42-e846cf41da2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:41.901208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800af5f6-f256-414c-be42-e846cf41da2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:41.902727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800af5f6-f256-414c-be42-e846cf41da2c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:41.905029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 65/132] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:41.911157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384f574-ba3f-4d19-a90b-f2d8ce5cd1c1 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:42.023198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384f574-ba3f-4d19-a90b-f2d8ce5cd1c1 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:42.057640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840bd48-1b27-427e-aa06-bb8e3912daaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:42.058277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840bd48-1b27-427e-aa06-bb8e3912daaa None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:42.058559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840bd48-1b27-427e-aa06-bb8e3912daaa None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:42.058831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840bd48-1b27-427e-aa06-bb8e3912daaa None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:42.059494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840bd48-1b27-427e-aa06-bb8e3912daaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:42.060962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840bd48-1b27-427e-aa06-bb8e3912daaa None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:42.070972 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:42.184636 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 68/133] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:11:41 2022] POST /identity/v3/users => generated 283 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:11:42.224520 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94949a-acdb-4fc0-9a29-95d9aaed2554 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:42.224805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94949a-acdb-4fc0-9a29-95d9aaed2554 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:42.225079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94949a-acdb-4fc0-9a29-95d9aaed2554 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:42.226486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 66/134] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:42.232229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb13fc-d4b0-4ba5-99b0-ef7fa17dabba None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:42.233265 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb13fc-d4b0-4ba5-99b0-ef7fa17dabba None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:42.233375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb13fc-d4b0-4ba5-99b0-ef7fa17dabba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:42.253382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e4cb13fc-d4b0-4ba5-99b0-ef7fa17dabba None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:42.254651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e4cb13fc-d4b0-4ba5-99b0-ef7fa17dabba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:42.324446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 69/135] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:42.595256 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94949a-acdb-4fc0-9a29-95d9aaed2554 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:42.684776 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:42.704679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94949a-acdb-4fc0-9a29-95d9aaed2554 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:42.743121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea5b9b-c514-42bc-b905-6b81c707568f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:42.743769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea5b9b-c514-42bc-b905-6b81c707568f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:42.744055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea5b9b-c514-42bc-b905-6b81c707568f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:42.744331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea5b9b-c514-42bc-b905-6b81c707568f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:42.756852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea5b9b-c514-42bc-b905-6b81c707568f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-feea5b9b-c514-42bc-b905-6b81c707568f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:42.762396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:42.763227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:42.764562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 67/136] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:42.770070 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb13fc-d4b0-4ba5-99b0-ef7fa17dabba None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:42.870739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb13fc-d4b0-4ba5-99b0-ef7fa17dabba None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:42.895482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:42.906143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05b8e4-cca6-4f86-932a-615e8eff077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:42.913622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05b8e4-cca6-4f86-932a-615e8eff077b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:42.913622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05b8e4-cca6-4f86-932a-615e8eff077b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:42.913622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05b8e4-cca6-4f86-932a-615e8eff077b None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:42.913622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05b8e4-cca6-4f86-932a-615e8eff077b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:42.913622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05b8e4-cca6-4f86-932a-615e8eff077b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:42.913622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05b8e4-cca6-4f86-932a-615e8eff077b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:42.918260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 70/137] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:42.924498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea5b9b-c514-42bc-b905-6b81c707568f None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:43.035168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea5b9b-c514-42bc-b905-6b81c707568f None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:43.069709 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e82b5-973b-47b5-9e0b-8a27816f2056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:43.070377 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e82b5-973b-47b5-9e0b-8a27816f2056 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:43.070663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e82b5-973b-47b5-9e0b-8a27816f2056 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:43.070938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e82b5-973b-47b5-9e0b-8a27816f2056 None admin] PATH_INFO: `/v3/users/19fa4560197a46b5bff8ec65cd3f6d84` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:43.072095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e82b5-973b-47b5-9e0b-8a27816f2056 None admin] RBAC: Authorizing `identity:get_user(user_id=19fa4560197a46b5bff8ec65cd3f6d84)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:43.073549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e82b5-973b-47b5-9e0b-8a27816f2056 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:43.075324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 68/138] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:42 2022] GET /identity/v3/users/19fa4560197a46b5bff8ec65cd3f6d84 => generated 289 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:43.081395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05b8e4-cca6-4f86-932a-615e8eff077b None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:43.190246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05b8e4-cca6-4f86-932a-615e8eff077b None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:43.225988 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4d9dd-24b8-4e33-a46b-7259623ae21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:43.226671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4d9dd-24b8-4e33-a46b-7259623ae21f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:43.226955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4d9dd-24b8-4e33-a46b-7259623ae21f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:43.227228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4d9dd-24b8-4e33-a46b-7259623ae21f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:43.283940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:43.297244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4d9dd-24b8-4e33-a46b-7259623ae21f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:43.298723 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4d9dd-24b8-4e33-a46b-7259623ae21f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:43.408560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 71/139] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:11:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:11:43.414216 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e82b5-973b-47b5-9e0b-8a27816f2056 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:43.427538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:43.522962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e82b5-973b-47b5-9e0b-8a27816f2056 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:43.556898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e77d1-9564-49aa-99ed-03a1be4c29ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:43.557533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e77d1-9564-49aa-99ed-03a1be4c29ac None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:43.557817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e77d1-9564-49aa-99ed-03a1be4c29ac None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:43.558118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e77d1-9564-49aa-99ed-03a1be4c29ac None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:43.559261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e77d1-9564-49aa-99ed-03a1be4c29ac None admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:43.560732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e77d1-9564-49aa-99ed-03a1be4c29ac None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:43.562406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 69/140] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:43 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:43.568508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4d9dd-24b8-4e33-a46b-7259623ae21f None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:43.655971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:43.672079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4d9dd-24b8-4e33-a46b-7259623ae21f None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:43.744505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c04865-422b-42e5-a01a-5c555b6a46b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:43.745152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c04865-422b-42e5-a01a-5c555b6a46b7 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:43.745426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c04865-422b-42e5-a01a-5c555b6a46b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:43.745738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c04865-422b-42e5-a01a-5c555b6a46b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:43.746504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c04865-422b-42e5-a01a-5c555b6a46b7 None admin] RBAC: Adding query filter params (role.id=dda0ba34c8b9442d9e12b1c8ab9a5d6c, scope.project.id=dce320a8ccf64f479c1dae1117865baf, user.id=19fa4560197a46b5bff8ec65cd3f6d84) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:43.746806 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c04865-422b-42e5-a01a-5c555b6a46b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:43.748260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c04865-422b-42e5-a01a-5c555b6a46b7 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:43.754404 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 72/141] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:11:43 2022] GET /identity/v3/role_assignments?user.id=19fa4560197a46b5bff8ec65cd3f6d84&scope.project.id=dce320a8ccf64f479c1dae1117865baf&role.id=dda0ba34c8b9442d9e12b1c8ab9a5d6c => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:44.455537 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5deb-33cd-4157-aa57-8f45c705563a None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:44.456098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5deb-33cd-4157-aa57-8f45c705563a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:44.456098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5deb-33cd-4157-aa57-8f45c705563a None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:44.457382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 70/142] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:44.462651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896a502-7d99-4687-86b4-3e28e3180cc2 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:44.463038 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896a502-7d99-4687-86b4-3e28e3180cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:44.463200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896a502-7d99-4687-86b4-3e28e3180cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:44.481046 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-2896a502-7d99-4687-86b4-3e28e3180cc2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:44.482250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2896a502-7d99-4687-86b4-3e28e3180cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:44.549524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 73/143] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:44.684620 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592b9b-95b7-406a-9c18-7cd8ab40f945 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:44.684988 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592b9b-95b7-406a-9c18-7cd8ab40f945 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:44.685179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24592b9b-95b7-406a-9c18-7cd8ab40f945 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:44.686701 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 71/144] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:44.687231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa234-c865-4902-bfb8-9b40222c0c0a None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:44.687523 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa234-c865-4902-bfb8-9b40222c0c0a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:44.687827 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa234-c865-4902-bfb8-9b40222c0c0a None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:44.689253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 74/145] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:11:44.691582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca042-6231-4cf9-865d-6a0e041c0ac7 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:44.691866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca042-6231-4cf9-865d-6a0e041c0ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:44.692141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca042-6231-4cf9-865d-6a0e041c0ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:44.693751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbab4b-e99a-4031-a1a7-bc2e4ab78b87 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:44.694038 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbab4b-e99a-4031-a1a7-bc2e4ab78b87 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:44.694340 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbab4b-e99a-4031-a1a7-bc2e4ab78b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:44.710025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-1e8ca042-6231-4cf9-865d-6a0e041c0ac7 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:44.711517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1e8ca042-6231-4cf9-865d-6a0e041c0ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:44.711948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-d7bbab4b-e99a-4031-a1a7-bc2e4ab78b87 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:44.713083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d7bbab4b-e99a-4031-a1a7-bc2e4ab78b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:44.782902 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 72/146] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:44.784285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 75/147] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:11:44.823338 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ca042-6231-4cf9-865d-6a0e041c0ac7 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:44.834406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:44.922671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ca042-6231-4cf9-865d-6a0e041c0ac7 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:44.956812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f45f05-3efa-4994-87c0-a905db251c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:44.957441 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f45f05-3efa-4994-87c0-a905db251c48 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:44.957723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f45f05-3efa-4994-87c0-a905db251c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:44.957997 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f45f05-3efa-4994-87c0-a905db251c48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:44.968470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f45f05-3efa-4994-87c0-a905db251c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-40f45f05-3efa-4994-87c0-a905db251c48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:44.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:44.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:44.975514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:44.978344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 73/148] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:11:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:44.980363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbab4b-e99a-4031-a1a7-bc2e4ab78b87 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:44.992393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:45.051380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f45f05-3efa-4994-87c0-a905db251c48 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:45.078550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbab4b-e99a-4031-a1a7-bc2e4ab78b87 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:45.114943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8fcfa-a160-4e13-8a24-974146e4060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:45.115542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8fcfa-a160-4e13-8a24-974146e4060b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:45.115813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8fcfa-a160-4e13-8a24-974146e4060b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:45.116086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8fcfa-a160-4e13-8a24-974146e4060b None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:45.116737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8fcfa-a160-4e13-8a24-974146e4060b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:45.117035 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8fcfa-a160-4e13-8a24-974146e4060b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:45.118424 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8fcfa-a160-4e13-8a24-974146e4060b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:45.120662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:45.127085 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 76/149] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:45.128394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8fcfa-a160-4e13-8a24-974146e4060b None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:45.153348 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f45f05-3efa-4994-87c0-a905db251c48 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:45.186784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9002a-3bec-44ce-a19f-c4f2e37be44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:45.187409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9002a-3bec-44ce-a19f-c4f2e37be44c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:45.187685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9002a-3bec-44ce-a19f-c4f2e37be44c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:45.187954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc9002a-3bec-44ce-a19f-c4f2e37be44c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:45.195698 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:45.197561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc9002a-3bec-44ce-a19f-c4f2e37be44c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-6fc9002a-3bec-44ce-a19f-c4f2e37be44c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:45.203118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:45.204241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:45.205152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 74/150] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:11:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:45.206912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9002a-3bec-44ce-a19f-c4f2e37be44c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:45.224753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8fcfa-a160-4e13-8a24-974146e4060b None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:45.257746 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2c4e9-2306-4ccb-9334-a999b78cdd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:45.258599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2c4e9-2306-4ccb-9334-a999b78cdd39 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:45.259042 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2c4e9-2306-4ccb-9334-a999b78cdd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:45.259317 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2c4e9-2306-4ccb-9334-a999b78cdd39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:45.260411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2c4e9-2306-4ccb-9334-a999b78cdd39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:45.261849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2c4e9-2306-4ccb-9334-a999b78cdd39 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:45.263595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 77/151] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:45.264282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2c4e9-2306-4ccb-9334-a999b78cdd39 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:45.303252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc9002a-3bec-44ce-a19f-c4f2e37be44c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:45.327955 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:45.335307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669781-3298-4402-b5f4-a9d39649e91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:45.335941 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669781-3298-4402-b5f4-a9d39649e91d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:45.336478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669781-3298-4402-b5f4-a9d39649e91d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:45.336753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b669781-3298-4402-b5f4-a9d39649e91d None admin] PATH_INFO: `/v3/users/317105b43c5e458180feb80affcd97bb` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:45.349989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b669781-3298-4402-b5f4-a9d39649e91d None admin] RBAC: Authorizing `identity:get_user(user_id=317105b43c5e458180feb80affcd97bb)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:45.351484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b669781-3298-4402-b5f4-a9d39649e91d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:45.353082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 75/152] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:45 2022] GET /identity/v3/users/317105b43c5e458180feb80affcd97bb => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:45.353819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669781-3298-4402-b5f4-a9d39649e91d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:45.362096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2c4e9-2306-4ccb-9334-a999b78cdd39 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:45.401532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c29cb-083e-4794-874a-57cfdaa61d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:45.402164 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c29cb-083e-4794-874a-57cfdaa61d53 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:45.402448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c29cb-083e-4794-874a-57cfdaa61d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:45.402718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c29cb-083e-4794-874a-57cfdaa61d53 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:45.403375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c29cb-083e-4794-874a-57cfdaa61d53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:45.403674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c29cb-083e-4794-874a-57cfdaa61d53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:45.405610 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2c29cb-083e-4794-874a-57cfdaa61d53 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:45.413407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 78/153] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:11:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:45.414299 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c29cb-083e-4794-874a-57cfdaa61d53 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:45.416777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:45.468612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b669781-3298-4402-b5f4-a9d39649e91d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:45.502134 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62ebf0-564a-4c64-9886-3bf335388e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:45.502738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62ebf0-564a-4c64-9886-3bf335388e91 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:45.503016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62ebf0-564a-4c64-9886-3bf335388e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:45.503286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62ebf0-564a-4c64-9886-3bf335388e91 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:45.503947 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62ebf0-564a-4c64-9886-3bf335388e91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:45.505369 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62ebf0-564a-4c64-9886-3bf335388e91 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:45.510063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c29cb-083e-4794-874a-57cfdaa61d53 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:45.542212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0e12f-6265-4b7f-b162-9d59ccdea6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:45.542824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0e12f-6265-4b7f-b162-9d59ccdea6ff None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:45.543097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0e12f-6265-4b7f-b162-9d59ccdea6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:45.543365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0e12f-6265-4b7f-b162-9d59ccdea6ff None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:45.544481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0e12f-6265-4b7f-b162-9d59ccdea6ff None admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:45.545946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0e12f-6265-4b7f-b162-9d59ccdea6ff None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:45.547567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 79/154] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:45 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:45.548307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0e12f-6265-4b7f-b162-9d59ccdea6ff None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:45.603000 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 76/155] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:11:45 2022] POST /identity/v3/users => generated 297 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:45.603805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62ebf0-564a-4c64-9886-3bf335388e91 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:45.620007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:45.652888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:45.661360 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0e12f-6265-4b7f-b162-9d59ccdea6ff None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:45.703329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a63bca-051d-4d73-bb3e-e597bdcaf2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:45.704104 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a63bca-051d-4d73-bb3e-e597bdcaf2ad None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:45.704460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a63bca-051d-4d73-bb3e-e597bdcaf2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:45.704638 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a63bca-051d-4d73-bb3e-e597bdcaf2ad None admin] PATH_INFO: `/v3/users/317105b43c5e458180feb80affcd97bb` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:45.705801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a63bca-051d-4d73-bb3e-e597bdcaf2ad None admin] RBAC: Authorizing `identity:get_user(user_id=317105b43c5e458180feb80affcd97bb)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:45.707327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a63bca-051d-4d73-bb3e-e597bdcaf2ad None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:45.709181 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 80/156] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:45 2022] GET /identity/v3/users/317105b43c5e458180feb80affcd97bb => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:45.714662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a63bca-051d-4d73-bb3e-e597bdcaf2ad None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:45.717898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62ebf0-564a-4c64-9886-3bf335388e91 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:45.755021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166860c-812d-4515-829b-ad3d7517b3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:45.755625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166860c-812d-4515-829b-ad3d7517b3ae None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:45.755905 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166860c-812d-4515-829b-ad3d7517b3ae None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:45.756178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166860c-812d-4515-829b-ad3d7517b3ae None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b/users/317105b43c5e458180feb80affcd97bb/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:45.758246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166860c-812d-4515-829b-ad3d7517b3ae None admin] RBAC: Authorizing `identity:create_grant(project_id=b31fe422931242de9e41c5dc9136868b, user_id=317105b43c5e458180feb80affcd97bb, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:45.759727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166860c-812d-4515-829b-ad3d7517b3ae None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:45.819919 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a63bca-051d-4d73-bb3e-e597bdcaf2ad None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:45.834562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 77/157] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:11:45 2022] PUT /identity/v3/projects/b31fe422931242de9e41c5dc9136868b/users/317105b43c5e458180feb80affcd97bb/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:11:45.856905 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb9816-ddb4-4455-b143-eba0d6754acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:45.857555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb9816-ddb4-4455-b143-eba0d6754acd None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:45.857832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb9816-ddb4-4455-b143-eba0d6754acd None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:45.858170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb9816-ddb4-4455-b143-eba0d6754acd None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:45.859324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb9816-ddb4-4455-b143-eba0d6754acd None admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:45.860835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb9816-ddb4-4455-b143-eba0d6754acd None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:45.862807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 81/158] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:45 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:45.868765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166860c-812d-4515-829b-ad3d7517b3ae None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:45.999103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166860c-812d-4515-829b-ad3d7517b3ae None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:46.001086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:46.037453 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d38ba-81b7-493c-bbfd-6cb571afab39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:46.038065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d38ba-81b7-493c-bbfd-6cb571afab39 None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:46.038348 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d38ba-81b7-493c-bbfd-6cb571afab39 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:46.038626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d38ba-81b7-493c-bbfd-6cb571afab39 None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b/users/317105b43c5e458180feb80affcd97bb/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:46.040828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d38ba-81b7-493c-bbfd-6cb571afab39 None admin] RBAC: Authorizing `identity:create_grant(project_id=b31fe422931242de9e41c5dc9136868b, user_id=317105b43c5e458180feb80affcd97bb, role_id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:46.042350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d38ba-81b7-493c-bbfd-6cb571afab39 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:46.101357 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 78/159] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:11:45 2022] PUT /identity/v3/projects/b31fe422931242de9e41c5dc9136868b/users/317105b43c5e458180feb80affcd97bb/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:11:47.925994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed968c-c715-444b-bf7f-2a69ac3cd6be None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:47.926590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed968c-c715-444b-bf7f-2a69ac3cd6be None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:47.926590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed968c-c715-444b-bf7f-2a69ac3cd6be None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:47.928063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 82/160] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:47.933369 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51716c17-3c51-482a-8a30-893ac75bf664 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:47.933776 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51716c17-3c51-482a-8a30-893ac75bf664 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:47.933950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51716c17-3c51-482a-8a30-893ac75bf664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:47.958801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-51716c17-3c51-482a-8a30-893ac75bf664 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:47.963747 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-51716c17-3c51-482a-8a30-893ac75bf664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:47.995058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809748f-ca09-4a04-acae-8bc192fb4ca0 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:47.995376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809748f-ca09-4a04-acae-8bc192fb4ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:47.995605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809748f-ca09-4a04-acae-8bc192fb4ca0 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:47.997153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 83/161] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:48.015578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3d78d-265f-4d7d-adb3-9e10adefeae9 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:48.022080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3d78d-265f-4d7d-adb3-9e10adefeae9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:48.022080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3d78d-265f-4d7d-adb3-9e10adefeae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:48.044322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 79/162] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:48.063218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:48.069780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-bdc3d78d-265f-4d7d-adb3-9e10adefeae9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:48.071232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-bdc3d78d-265f-4d7d-adb3-9e10adefeae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:48.114545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474c2bb-3e05-4b01-8d86-a91ad791aff6 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:48.114915 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474c2bb-3e05-4b01-8d86-a91ad791aff6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:48.115191 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474c2bb-3e05-4b01-8d86-a91ad791aff6 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:48.120633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 80/163] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:48 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:48.123205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538a92-a84d-4fa3-8e52-cacb1e9b651f None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:48.123494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538a92-a84d-4fa3-8e52-cacb1e9b651f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:48.123772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79538a92-a84d-4fa3-8e52-cacb1e9b651f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:48.142731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 84/164] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:48.167513 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-79538a92-a84d-4fa3-8e52-cacb1e9b651f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:48.168715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-79538a92-a84d-4fa3-8e52-cacb1e9b651f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:48.254189 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 81/165] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:48.305786 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a49cf-8234-441a-b8f9-b9cdb4ca539c None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:48.306083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a49cf-8234-441a-b8f9-b9cdb4ca539c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:48.306365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a49cf-8234-441a-b8f9-b9cdb4ca539c None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:48.307722 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 85/166] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:48.313119 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8cb6-b615-424a-8172-0cfd8a493868 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:48.313407 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8cb6-b615-424a-8172-0cfd8a493868 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:48.313682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf8cb6-b615-424a-8172-0cfd8a493868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:48.347240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-4bcf8cb6-b615-424a-8172-0cfd8a493868 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:48.348427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4bcf8cb6-b615-424a-8172-0cfd8a493868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:48.370417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:48.410094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a49cf-8234-441a-b8f9-b9cdb4ca539c None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:48.450644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 82/167] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:48.452303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf8cb6-b615-424a-8172-0cfd8a493868 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:48.578805 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a49cf-8234-441a-b8f9-b9cdb4ca539c None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:48.588999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf8cb6-b615-424a-8172-0cfd8a493868 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:48.615417 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec82897-2e8a-4670-8245-4a3871785c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:48.616347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec82897-2e8a-4670-8245-4a3871785c6e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:48.616887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec82897-2e8a-4670-8245-4a3871785c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:48.617165 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec82897-2e8a-4670-8245-4a3871785c6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:48.620878 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:48.623737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04b434-58e0-4352-9c66-09262cbb6299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:48.624373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04b434-58e0-4352-9c66-09262cbb6299 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:48.624655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04b434-58e0-4352-9c66-09262cbb6299 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:48.624930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04b434-58e0-4352-9c66-09262cbb6299 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:48.626023 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04b434-58e0-4352-9c66-09262cbb6299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:48.627498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04b434-58e0-4352-9c66-09262cbb6299 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:48.633389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 83/168] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:48.633389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04b434-58e0-4352-9c66-09262cbb6299 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec82897-2e8a-4670-8245-4a3871785c6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-3ec82897-2e8a-4670-8245-4a3871785c6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:48.644200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:48.645090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:48.646161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 86/169] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:11:48.647119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec82897-2e8a-4670-8245-4a3871785c6e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:48.659462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:48.743766 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec82897-2e8a-4670-8245-4a3871785c6e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:48.745450 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04b434-58e0-4352-9c66-09262cbb6299 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:48.776579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6091d-9e89-4320-860f-1616f36c3596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:48.777372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6091d-9e89-4320-860f-1616f36c3596 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:48.777643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6091d-9e89-4320-860f-1616f36c3596 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:48.777909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed6091d-9e89-4320-860f-1616f36c3596 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:48.778588 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6091d-9e89-4320-860f-1616f36c3596 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:48.779028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0dae2-a85b-41c3-82fe-7a40a6b3adb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:48.779683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0dae2-a85b-41c3-82fe-7a40a6b3adb6 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:48.779959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0dae2-a85b-41c3-82fe-7a40a6b3adb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:48.780046 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed6091d-9e89-4320-860f-1616f36c3596 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:48.780230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0dae2-a85b-41c3-82fe-7a40a6b3adb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:48.790433 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0dae2-a85b-41c3-82fe-7a40a6b3adb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-dba0dae2-a85b-41c3-82fe-7a40a6b3adb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:48.795805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:48.796887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 84/170] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:11:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:48.797980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0dae2-a85b-41c3-82fe-7a40a6b3adb6 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:48.865762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 87/171] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:11:48 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:48.871230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6091d-9e89-4320-860f-1616f36c3596 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:48.877070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:48.897713 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0dae2-a85b-41c3-82fe-7a40a6b3adb6 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:48.932001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:48.938243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b30420-0dc6-4571-94d2-f212ecdb60e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:48.938884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b30420-0dc6-4571-94d2-f212ecdb60e1 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:48.939165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b30420-0dc6-4571-94d2-f212ecdb60e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:48.939442 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b30420-0dc6-4571-94d2-f212ecdb60e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:48.940130 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b30420-0dc6-4571-94d2-f212ecdb60e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:48.940439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b30420-0dc6-4571-94d2-f212ecdb60e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:48.941893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b30420-0dc6-4571-94d2-f212ecdb60e1 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:48.955976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 85/172] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:48.956983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b30420-0dc6-4571-94d2-f212ecdb60e1 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:48.987736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed6091d-9e89-4320-860f-1616f36c3596 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:49.024814 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dac85-b3fc-4282-8d10-24f995900561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:49.026129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dac85-b3fc-4282-8d10-24f995900561 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:49.026647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dac85-b3fc-4282-8d10-24f995900561 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:49.027480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02dac85-b3fc-4282-8d10-24f995900561 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:49.038843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02dac85-b3fc-4282-8d10-24f995900561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-e02dac85-b3fc-4282-8d10-24f995900561 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:49.044253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:49.045768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 88/173] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:11:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:49.046792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dac85-b3fc-4282-8d10-24f995900561 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:49.083839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b30420-0dc6-4571-94d2-f212ecdb60e1 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:49.122334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46fb9-ae8b-4e61-ae6a-73cee82d872f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:49.123235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46fb9-ae8b-4e61-ae6a-73cee82d872f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:49.123520 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46fb9-ae8b-4e61-ae6a-73cee82d872f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:49.123799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db46fb9-ae8b-4e61-ae6a-73cee82d872f None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:49.124468 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46fb9-ae8b-4e61-ae6a-73cee82d872f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:49.124768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46fb9-ae8b-4e61-ae6a-73cee82d872f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:49.126330 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db46fb9-ae8b-4e61-ae6a-73cee82d872f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:49.139746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 86/174] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:49.140651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46fb9-ae8b-4e61-ae6a-73cee82d872f None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:49.146017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:49.167150 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02dac85-b3fc-4282-8d10-24f995900561 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:49.182507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:49.207564 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25193f-b125-423b-9a31-c27532d7fd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:49.208642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25193f-b125-423b-9a31-c27532d7fd74 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:49.209159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25193f-b125-423b-9a31-c27532d7fd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:49.209714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25193f-b125-423b-9a31-c27532d7fd74 None admin] PATH_INFO: `/v3/users/037b7a45699e4379bbf7860f59a6e08b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:49.227848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25193f-b125-423b-9a31-c27532d7fd74 None admin] RBAC: Authorizing `identity:get_user(user_id=037b7a45699e4379bbf7860f59a6e08b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:49.229343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25193f-b125-423b-9a31-c27532d7fd74 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:49.231623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 89/175] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:49 2022] GET /identity/v3/users/037b7a45699e4379bbf7860f59a6e08b => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:49.231996 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25193f-b125-423b-9a31-c27532d7fd74 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:49.274547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db46fb9-ae8b-4e61-ae6a-73cee82d872f None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:49.313257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bbc3e-42ac-4b4b-87a2-5c514bed7b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:49.314261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bbc3e-42ac-4b4b-87a2-5c514bed7b1e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:49.314791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bbc3e-42ac-4b4b-87a2-5c514bed7b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:49.315401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bbc3e-42ac-4b4b-87a2-5c514bed7b1e None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:49.316146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bbc3e-42ac-4b4b-87a2-5c514bed7b1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:49.316586 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bbc3e-42ac-4b4b-87a2-5c514bed7b1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:49.318074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bbc3e-42ac-4b4b-87a2-5c514bed7b1e None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:49.327922 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 87/176] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:11:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:49.333274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bbc3e-42ac-4b4b-87a2-5c514bed7b1e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:49.348250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25193f-b125-423b-9a31-c27532d7fd74 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:49.391144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa9852-6ddd-4391-9224-f6b19291152d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:49.392573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa9852-6ddd-4391-9224-f6b19291152d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:49.393307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa9852-6ddd-4391-9224-f6b19291152d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:49.393939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa9852-6ddd-4391-9224-f6b19291152d None admin] PATH_INFO: `/v3/users/317105b43c5e458180feb80affcd97bb` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:49.396006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa9852-6ddd-4391-9224-f6b19291152d None admin] RBAC: Authorizing `identity:get_user(user_id=317105b43c5e458180feb80affcd97bb)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:49.398004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa9852-6ddd-4391-9224-f6b19291152d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:49.400534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 90/177] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:49 2022] GET /identity/v3/users/317105b43c5e458180feb80affcd97bb => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:49.401748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa9852-6ddd-4391-9224-f6b19291152d None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:49.414493 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:49.459966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bbc3e-42ac-4b4b-87a2-5c514bed7b1e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:49.462217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:49.499942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7f997-d60b-4d69-bb5d-4179ed9b3586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:49.500851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7f997-d60b-4d69-bb5d-4179ed9b3586 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:49.501271 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7f997-d60b-4d69-bb5d-4179ed9b3586 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:49.501686 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc7f997-d60b-4d69-bb5d-4179ed9b3586 None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:49.503015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7f997-d60b-4d69-bb5d-4179ed9b3586 None admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:49.505590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc7f997-d60b-4d69-bb5d-4179ed9b3586 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:49.512994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 88/178] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:49 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:49.515273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7f997-d60b-4d69-bb5d-4179ed9b3586 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:49.530431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa9852-6ddd-4391-9224-f6b19291152d None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:49.566365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b824737-eb17-404f-8c6a-6a5f175a7925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:49.567214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b824737-eb17-404f-8c6a-6a5f175a7925 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:49.567644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b824737-eb17-404f-8c6a-6a5f175a7925 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:49.568018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b824737-eb17-404f-8c6a-6a5f175a7925 None admin] PATH_INFO: `/v3/users/317105b43c5e458180feb80affcd97bb` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:49.569363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b824737-eb17-404f-8c6a-6a5f175a7925 None admin] RBAC: Authorizing `identity:get_user(user_id=317105b43c5e458180feb80affcd97bb)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:49.571097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b824737-eb17-404f-8c6a-6a5f175a7925 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:49.572853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 91/179] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:49 2022] GET /identity/v3/users/317105b43c5e458180feb80affcd97bb => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:49.578532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b824737-eb17-404f-8c6a-6a5f175a7925 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:49.633774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc7f997-d60b-4d69-bb5d-4179ed9b3586 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:49.673341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0d4f0-fbce-4293-8399-43bcedba6a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:49.673882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0d4f0-fbce-4293-8399-43bcedba6a33 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:49.674185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0d4f0-fbce-4293-8399-43bcedba6a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:49.674457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0d4f0-fbce-4293-8399-43bcedba6a33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:49.678143 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:49.685376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b824737-eb17-404f-8c6a-6a5f175a7925 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:49.693213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:49.720670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22ac36-58dd-469b-abf6-2c933f621ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:49.721474 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22ac36-58dd-469b-abf6-2c933f621ba2 None admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:49.721872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22ac36-58dd-469b-abf6-2c933f621ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:49.722256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22ac36-58dd-469b-abf6-2c933f621ba2 None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b/users/037b7a45699e4379bbf7860f59a6e08b/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:49.724281 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22ac36-58dd-469b-abf6-2c933f621ba2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b31fe422931242de9e41c5dc9136868b, user_id=037b7a45699e4379bbf7860f59a6e08b, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:49.725838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22ac36-58dd-469b-abf6-2c933f621ba2 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:49.789224 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0d4f0-fbce-4293-8399-43bcedba6a33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:49.790709 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0d4f0-fbce-4293-8399-43bcedba6a33 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:49.810361 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 92/180] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:11:49 2022] PUT /identity/v3/projects/b31fe422931242de9e41c5dc9136868b/users/037b7a45699e4379bbf7860f59a6e08b/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:11:49.813815 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22ac36-58dd-469b-abf6-2c933f621ba2 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:49.992282 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 89/181] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:11:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:11:49.995049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22ac36-58dd-469b-abf6-2c933f621ba2 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:49.997834 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0d4f0-fbce-4293-8399-43bcedba6a33 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:50.044092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b4030-b6fd-4ba1-b554-412175503e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:50.044702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b4030-b6fd-4ba1-b554-412175503e3f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:50.045040 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b4030-b6fd-4ba1-b554-412175503e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:50.045334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b4030-b6fd-4ba1-b554-412175503e3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:50.046127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:50.091535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:50.136156 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b4030-b6fd-4ba1-b554-412175503e3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:50.138031 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b4030-b6fd-4ba1-b554-412175503e3f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:50.148728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0d4f0-fbce-4293-8399-43bcedba6a33 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:50.221331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb99c2-b466-4c7a-87c5-6730033cd568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:50.224194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb99c2-b466-4c7a-87c5-6730033cd568 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:50.224473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb99c2-b466-4c7a-87c5-6730033cd568 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:50.224744 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb99c2-b466-4c7a-87c5-6730033cd568 None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:50.226684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb99c2-b466-4c7a-87c5-6730033cd568 None admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:50.228976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cb99c2-b466-4c7a-87c5-6730033cd568 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:50.230654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 90/182] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:49 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:50.237408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb99c2-b466-4c7a-87c5-6730033cd568 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:50.310608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 93/183] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:11:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:11:50.316732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b4030-b6fd-4ba1-b554-412175503e3f None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:50.400348 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cb99c2-b466-4c7a-87c5-6730033cd568 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:50.401845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:50.437841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288d4cf-3537-4ea4-b400-c4eac3e6fe65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:50.438484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288d4cf-3537-4ea4-b400-c4eac3e6fe65 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:50.438760 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288d4cf-3537-4ea4-b400-c4eac3e6fe65 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:50.439029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288d4cf-3537-4ea4-b400-c4eac3e6fe65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:50.439767 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288d4cf-3537-4ea4-b400-c4eac3e6fe65 None admin] RBAC: Adding query filter params (role.id=d5070bb151d343ca8de32abe373f4a73, scope.project.id=b31fe422931242de9e41c5dc9136868b, user.id=317105b43c5e458180feb80affcd97bb) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:50.440076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288d4cf-3537-4ea4-b400-c4eac3e6fe65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:50.441700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288d4cf-3537-4ea4-b400-c4eac3e6fe65 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:50.454382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:50.456394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 91/184] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:11:50 2022] GET /identity/v3/role_assignments?user.id=317105b43c5e458180feb80affcd97bb&scope.project.id=b31fe422931242de9e41c5dc9136868b&role.id=d5070bb151d343ca8de32abe373f4a73 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:50.492405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b4030-b6fd-4ba1-b554-412175503e3f None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:50.543400 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617aaab1-9e7a-4c9b-9f7a-5c4ca90a5569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:50.544052 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aaab1-9e7a-4c9b-9f7a-5c4ca90a5569 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:50.544327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aaab1-9e7a-4c9b-9f7a-5c4ca90a5569 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:50.544595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aaab1-9e7a-4c9b-9f7a-5c4ca90a5569 None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:50.546405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617aaab1-9e7a-4c9b-9f7a-5c4ca90a5569 None admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:50.547891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617aaab1-9e7a-4c9b-9f7a-5c4ca90a5569 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:50.549594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 94/185] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:50 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:50.555726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288d4cf-3537-4ea4-b400-c4eac3e6fe65 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:50.667300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288d4cf-3537-4ea4-b400-c4eac3e6fe65 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:50.702724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d61713-d7f0-42ee-a652-d1290f2f4db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:50.703324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d61713-d7f0-42ee-a652-d1290f2f4db3 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:50.703636 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d61713-d7f0-42ee-a652-d1290f2f4db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:50.703908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d61713-d7f0-42ee-a652-d1290f2f4db3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:50.704638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d61713-d7f0-42ee-a652-d1290f2f4db3 None admin] RBAC: Adding query filter params (role.id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6, scope.project.id=b31fe422931242de9e41c5dc9136868b, user.id=317105b43c5e458180feb80affcd97bb) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:50.704938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d61713-d7f0-42ee-a652-d1290f2f4db3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:50.706478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d61713-d7f0-42ee-a652-d1290f2f4db3 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:50.712807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 92/186] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:11:50 2022] GET /identity/v3/role_assignments?user.id=317105b43c5e458180feb80affcd97bb&scope.project.id=b31fe422931242de9e41c5dc9136868b&role.id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:51.255550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae27219-14cf-4eb5-bfbd-8e278cd59793 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:51.256080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae27219-14cf-4eb5-bfbd-8e278cd59793 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:51.256080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae27219-14cf-4eb5-bfbd-8e278cd59793 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:51.257479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 95/187] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:51.262567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666ed9-deb5-4f89-8b9d-02a0bbbc55ff None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:51.262978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666ed9-deb5-4f89-8b9d-02a0bbbc55ff None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:51.263139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666ed9-deb5-4f89-8b9d-02a0bbbc55ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:51.282343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c2666ed9-deb5-4f89-8b9d-02a0bbbc55ff None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:51.283518 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c2666ed9-deb5-4f89-8b9d-02a0bbbc55ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:51.307536 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:51.352321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 93/188] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:51.467970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f35cd-443a-4885-9713-31cbdd7a2cda None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:51.468266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f35cd-443a-4885-9713-31cbdd7a2cda None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:51.468440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f35cd-443a-4885-9713-31cbdd7a2cda None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:51.469922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 96/189] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:51.474918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222510f-2b69-4ef9-9031-c7c423b5beda None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:51.475208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222510f-2b69-4ef9-9031-c7c423b5beda None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:51.475483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222510f-2b69-4ef9-9031-c7c423b5beda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:51.493347 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2222510f-2b69-4ef9-9031-c7c423b5beda None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:51.494539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2222510f-2b69-4ef9-9031-c7c423b5beda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:51.560158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 94/190] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:51.617700 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f35cd-443a-4885-9713-31cbdd7a2cda None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:51.751558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f35cd-443a-4885-9713-31cbdd7a2cda None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:51.785403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38381c86-aa81-4c0e-ac7b-b0d7c5f6d032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:51.786017 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38381c86-aa81-4c0e-ac7b-b0d7c5f6d032 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:51.786315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38381c86-aa81-4c0e-ac7b-b0d7c5f6d032 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:51.786579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38381c86-aa81-4c0e-ac7b-b0d7c5f6d032 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:51.788465 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:51.797432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38381c86-aa81-4c0e-ac7b-b0d7c5f6d032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-38381c86-aa81-4c0e-ac7b-b0d7c5f6d032 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:51.802324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:51.803189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:51.803950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 97/191] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:51.810470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222510f-2b69-4ef9-9031-c7c423b5beda None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:51.828019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38381c86-aa81-4c0e-ac7b-b0d7c5f6d032 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:51.909314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:51.918900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222510f-2b69-4ef9-9031-c7c423b5beda None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:51.937578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38381c86-aa81-4c0e-ac7b-b0d7c5f6d032 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:51.954477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49482-316e-478e-bfc6-903a6a47c32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:51.955087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49482-316e-478e-bfc6-903a6a47c32f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:51.955366 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49482-316e-478e-bfc6-903a6a47c32f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:51.955635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a49482-316e-478e-bfc6-903a6a47c32f None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:51.956297 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a49482-316e-478e-bfc6-903a6a47c32f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:51.956595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a49482-316e-478e-bfc6-903a6a47c32f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:51.957966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a49482-316e-478e-bfc6-903a6a47c32f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:51.966448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 95/192] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:51.972170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49482-316e-478e-bfc6-903a6a47c32f None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:51.978199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cf626-008b-44d6-8da1-b265dd337e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:51.978839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf626-008b-44d6-8da1-b265dd337e86 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:51.979115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf626-008b-44d6-8da1-b265dd337e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:51.979384 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cf626-008b-44d6-8da1-b265dd337e86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:51.980914 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cf626-008b-44d6-8da1-b265dd337e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:51.982509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cf626-008b-44d6-8da1-b265dd337e86 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:51.984985 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 98/193] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:51.990135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cf626-008b-44d6-8da1-b265dd337e86 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:52.054899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:52.080442 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a49482-316e-478e-bfc6-903a6a47c32f None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:52.094064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cf626-008b-44d6-8da1-b265dd337e86 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:52.126183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6eb5d9-dd96-4667-99f6-9686ee3c5201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:52.126867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6eb5d9-dd96-4667-99f6-9686ee3c5201 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:52.127096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6eb5d9-dd96-4667-99f6-9686ee3c5201 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:52.127406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6eb5d9-dd96-4667-99f6-9686ee3c5201 None admin] PATH_INFO: `/v3/users/1939a09058d3432788297f8a2f2ea776` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:52.129753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831a2f7-b41c-4d9d-9d30-227fbe5d7dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:52.130398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831a2f7-b41c-4d9d-9d30-227fbe5d7dae None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:52.130675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831a2f7-b41c-4d9d-9d30-227fbe5d7dae None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:52.130948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831a2f7-b41c-4d9d-9d30-227fbe5d7dae None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:52.131622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831a2f7-b41c-4d9d-9d30-227fbe5d7dae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:52.133105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831a2f7-b41c-4d9d-9d30-227fbe5d7dae None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:52.143281 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6eb5d9-dd96-4667-99f6-9686ee3c5201 None admin] RBAC: Authorizing `identity:get_user(user_id=1939a09058d3432788297f8a2f2ea776)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:52.144973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6eb5d9-dd96-4667-99f6-9686ee3c5201 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:52.146720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 96/194] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:51 2022] GET /identity/v3/users/1939a09058d3432788297f8a2f2ea776 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:52.152486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6eb5d9-dd96-4667-99f6-9686ee3c5201 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:52.166565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:52.243387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 99/195] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:11:51 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:11:52.251380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6eb5d9-dd96-4667-99f6-9686ee3c5201 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:52.253791 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dae8f5-3a30-4add-94d8-9bedb205437c None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:52.254116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dae8f5-3a30-4add-94d8-9bedb205437c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:52.254400 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dae8f5-3a30-4add-94d8-9bedb205437c None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:52.255807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 100/196] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:11:52.261830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c3ac4-7c79-496e-bd6d-625be7bd9191 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:52.262155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c3ac4-7c79-496e-bd6d-625be7bd9191 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:52.262438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c3ac4-7c79-496e-bd6d-625be7bd9191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:52.282209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-613c3ac4-7c79-496e-bd6d-625be7bd9191 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:52.283457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-613c3ac4-7c79-496e-bd6d-625be7bd9191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:52.294364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16089ec3-f735-43ed-bef2-3130b9179573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:52.295728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16089ec3-f735-43ed-bef2-3130b9179573 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:52.296240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16089ec3-f735-43ed-bef2-3130b9179573 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:52.296777 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16089ec3-f735-43ed-bef2-3130b9179573 None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:52.298347 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16089ec3-f735-43ed-bef2-3130b9179573 None admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:52.300103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16089ec3-f735-43ed-bef2-3130b9179573 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:52.302315 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 97/197] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:52 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:52.308111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16089ec3-f735-43ed-bef2-3130b9179573 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:52.365398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 101/198] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:11:52.405012 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:52.426865 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16089ec3-f735-43ed-bef2-3130b9179573 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:52.464613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86871b-8192-4386-a5f4-b7d30d93be90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:52.465267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86871b-8192-4386-a5f4-b7d30d93be90 None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:52.465540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86871b-8192-4386-a5f4-b7d30d93be90 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:52.465809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86871b-8192-4386-a5f4-b7d30d93be90 None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b/users/1939a09058d3432788297f8a2f2ea776/roles/dda0ba34c8b9442d9e12b1c8ab9a5d6c` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:52.468274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86871b-8192-4386-a5f4-b7d30d93be90 None admin] RBAC: Authorizing `identity:create_grant(project_id=b31fe422931242de9e41c5dc9136868b, user_id=1939a09058d3432788297f8a2f2ea776, role_id=dda0ba34c8b9442d9e12b1c8ab9a5d6c)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:52.469695 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86871b-8192-4386-a5f4-b7d30d93be90 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:52.574794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 98/199] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:11:52 2022] PUT /identity/v3/projects/b31fe422931242de9e41c5dc9136868b/users/1939a09058d3432788297f8a2f2ea776/roles/dda0ba34c8b9442d9e12b1c8ab9a5d6c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:11:52.633876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c3ac4-7c79-496e-bd6d-625be7bd9191 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:52.688700 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:52.748582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c3ac4-7c79-496e-bd6d-625be7bd9191 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:52.789189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f4613-f01c-43d6-ac4c-1fa9b18c4175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:52.790102 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f4613-f01c-43d6-ac4c-1fa9b18c4175 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:52.790552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f4613-f01c-43d6-ac4c-1fa9b18c4175 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:52.790978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f4613-f01c-43d6-ac4c-1fa9b18c4175 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:52.802442 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f4613-f01c-43d6-ac4c-1fa9b18c4175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-d91f4613-f01c-43d6-ac4c-1fa9b18c4175 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:52.807475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:52.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:52.809570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:52.811902 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 102/200] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:52.817806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86871b-8192-4386-a5f4-b7d30d93be90 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:52.927158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86871b-8192-4386-a5f4-b7d30d93be90 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:52.962133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0afe7-af0d-4fac-8fbe-ec56de4ac545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:52.962746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0afe7-af0d-4fac-8fbe-ec56de4ac545 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:52.963023 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0afe7-af0d-4fac-8fbe-ec56de4ac545 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:52.963294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0afe7-af0d-4fac-8fbe-ec56de4ac545 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:52.963957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0afe7-af0d-4fac-8fbe-ec56de4ac545 None admin] RBAC: Adding query filter params (name=member) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:52.964255 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0afe7-af0d-4fac-8fbe-ec56de4ac545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:52.965618 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0afe7-af0d-4fac-8fbe-ec56de4ac545 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:52.971719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:52.975058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 99/201] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:52.980755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f4613-f01c-43d6-ac4c-1fa9b18c4175 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:53.086023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f4613-f01c-43d6-ac4c-1fa9b18c4175 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:53.096788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:53.122864 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66222d25-1b6e-4d70-ab42-505dcd30bf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:53.124023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66222d25-1b6e-4d70-ab42-505dcd30bf46 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:53.124293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66222d25-1b6e-4d70-ab42-505dcd30bf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:53.124556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66222d25-1b6e-4d70-ab42-505dcd30bf46 None admin] PATH_INFO: `/v3/users/037b7a45699e4379bbf7860f59a6e08b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:53.125689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66222d25-1b6e-4d70-ab42-505dcd30bf46 None admin] RBAC: Authorizing `identity:get_user(user_id=037b7a45699e4379bbf7860f59a6e08b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:53.127171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66222d25-1b6e-4d70-ab42-505dcd30bf46 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:53.129099 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 103/202] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:52 2022] GET /identity/v3/users/037b7a45699e4379bbf7860f59a6e08b => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:53.134898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0afe7-af0d-4fac-8fbe-ec56de4ac545 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:53.242025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0afe7-af0d-4fac-8fbe-ec56de4ac545 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:53.276640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc168d3-6699-434b-8c98-41104d3a8f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:53.277252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc168d3-6699-434b-8c98-41104d3a8f4c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:53.277529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc168d3-6699-434b-8c98-41104d3a8f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:53.277803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc168d3-6699-434b-8c98-41104d3a8f4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:53.349421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:53.352328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc168d3-6699-434b-8c98-41104d3a8f4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:53.353933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc168d3-6699-434b-8c98-41104d3a8f4c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:53.455960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 100/203] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:11:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:11:53.461877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66222d25-1b6e-4d70-ab42-505dcd30bf46 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:53.577155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66222d25-1b6e-4d70-ab42-505dcd30bf46 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:53.611614 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8e553-c2ba-483a-a47c-f5e31ec4367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:53.612242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8e553-c2ba-483a-a47c-f5e31ec4367f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:53.612514 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8e553-c2ba-483a-a47c-f5e31ec4367f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:53.612781 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8e553-c2ba-483a-a47c-f5e31ec4367f None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:53.613886 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8e553-c2ba-483a-a47c-f5e31ec4367f None admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:53.615352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8e553-c2ba-483a-a47c-f5e31ec4367f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:53.617138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 104/204] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:53 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:53.622816 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc168d3-6699-434b-8c98-41104d3a8f4c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:53.732374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:53.735717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc168d3-6699-434b-8c98-41104d3a8f4c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:53.770155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030de7d8-6b2b-49f5-bc95-9437d77d3a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:53.770756 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030de7d8-6b2b-49f5-bc95-9437d77d3a68 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:53.771031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030de7d8-6b2b-49f5-bc95-9437d77d3a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:53.771354 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030de7d8-6b2b-49f5-bc95-9437d77d3a68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:53.772079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030de7d8-6b2b-49f5-bc95-9437d77d3a68 None admin] RBAC: Adding query filter params (role.id=bb36a823b2ff45d29018aedf19ae7bf0, scope.project.id=b31fe422931242de9e41c5dc9136868b, user.id=037b7a45699e4379bbf7860f59a6e08b) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:53.772379 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030de7d8-6b2b-49f5-bc95-9437d77d3a68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:53.773817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030de7d8-6b2b-49f5-bc95-9437d77d3a68 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:53.786330 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 101/205] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:11:53 2022] GET /identity/v3/role_assignments?user.id=037b7a45699e4379bbf7860f59a6e08b&scope.project.id=b31fe422931242de9e41c5dc9136868b&role.id=bb36a823b2ff45d29018aedf19ae7bf0 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:54.573583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd5dce-8fce-4f1d-952a-a4981defc3d8 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:54.574140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd5dce-8fce-4f1d-952a-a4981defc3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:54.574140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd5dce-8fce-4f1d-952a-a4981defc3d8 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:54.575769 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 105/206] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:54.581477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e00a4-cabc-4d58-818a-bafcde54a659 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:54.581971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e00a4-cabc-4d58-818a-bafcde54a659 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:54.582226 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e00a4-cabc-4d58-818a-bafcde54a659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:54.601679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-f63e00a4-cabc-4d58-818a-bafcde54a659 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:54.602912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f63e00a4-cabc-4d58-818a-bafcde54a659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:54.667173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 102/207] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:54.691123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa2bef-3158-445f-904b-5e367300d8fd None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:54.691448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa2bef-3158-445f-904b-5e367300d8fd None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:54.691693 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa2bef-3158-445f-904b-5e367300d8fd None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:54.693272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 106/208] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:54.699178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd5bc0-c48a-4975-8d16-6997122b26f3 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:54.699470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd5bc0-c48a-4975-8d16-6997122b26f3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:54.699739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd5bc0-c48a-4975-8d16-6997122b26f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:54.719752 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-dabd5bc0-c48a-4975-8d16-6997122b26f3 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:54.720948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-dabd5bc0-c48a-4975-8d16-6997122b26f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:54.790775 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 103/209] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:54.799946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39da0da-4a4b-4ab5-9079-7c6a01ce0bbf None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:54.800233 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39da0da-4a4b-4ab5-9079-7c6a01ce0bbf None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:54.800508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39da0da-4a4b-4ab5-9079-7c6a01ce0bbf None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:54.801984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 107/210] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:54.806850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e4991-668c-4638-8b8c-a1af6f913661 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:54.807137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e4991-668c-4638-8b8c-a1af6f913661 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:54.807413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e4991-668c-4638-8b8c-a1af6f913661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:54.825488 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-a53e4991-668c-4638-8b8c-a1af6f913661 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:54.826650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a53e4991-668c-4638-8b8c-a1af6f913661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:54.843629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:54.891654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 104/211] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:54.930987 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39da0da-4a4b-4ab5-9079-7c6a01ce0bbf None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:54.977072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:55.030994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39da0da-4a4b-4ab5-9079-7c6a01ce0bbf None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:55.064547 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d25a5c-bfd7-4294-8a3a-801eded19fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:55.065166 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d25a5c-bfd7-4294-8a3a-801eded19fcf None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:55.065440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d25a5c-bfd7-4294-8a3a-801eded19fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:55.065707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d25a5c-bfd7-4294-8a3a-801eded19fcf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:55.075661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d25a5c-bfd7-4294-8a3a-801eded19fcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:55.080179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e4991-668c-4638-8b8c-a1af6f913661 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-55d25a5c-bfd7-4294-8a3a-801eded19fcf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:55.080875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:55.081706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:55.082858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:55.086189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 108/212] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:55.087666 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d25a5c-bfd7-4294-8a3a-801eded19fcf None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:55.191629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e4991-668c-4638-8b8c-a1af6f913661 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:55.192296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d25a5c-bfd7-4294-8a3a-801eded19fcf None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:55.194623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:55.225040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2c2f7-ae78-46bd-8096-49b9dc995c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:55.225663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2c2f7-ae78-46bd-8096-49b9dc995c97 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:55.225859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4aa53-018f-44fd-8089-2fd9c0709745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:55.225966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2c2f7-ae78-46bd-8096-49b9dc995c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:55.226225 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2c2f7-ae78-46bd-8096-49b9dc995c97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:55.226477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4aa53-018f-44fd-8089-2fd9c0709745 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:55.226755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4aa53-018f-44fd-8089-2fd9c0709745 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:55.227024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4aa53-018f-44fd-8089-2fd9c0709745 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:55.227311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2c2f7-ae78-46bd-8096-49b9dc995c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:55.227672 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4aa53-018f-44fd-8089-2fd9c0709745 None admin] RBAC: Adding query filter params (name=member) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:55.227970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4aa53-018f-44fd-8089-2fd9c0709745 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:55.228801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2c2f7-ae78-46bd-8096-49b9dc995c97 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:55.229330 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c4aa53-018f-44fd-8089-2fd9c0709745 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:55.230412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 105/213] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:55.230412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2c2f7-ae78-46bd-8096-49b9dc995c97 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:55.243096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 109/214] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:55.243914 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4aa53-018f-44fd-8089-2fd9c0709745 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:55.258669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:55.333152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2c2f7-ae78-46bd-8096-49b9dc995c97 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:55.340359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4aa53-018f-44fd-8089-2fd9c0709745 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:55.366200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f797f0-ed3c-46d0-9f89-175a28336724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:55.366819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f797f0-ed3c-46d0-9f89-175a28336724 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:55.367093 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f797f0-ed3c-46d0-9f89-175a28336724 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:55.367356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f797f0-ed3c-46d0-9f89-175a28336724 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:55.371850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50163d88-03a7-47ed-be51-84280b55872b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:55.372525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50163d88-03a7-47ed-be51-84280b55872b None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:55.372750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50163d88-03a7-47ed-be51-84280b55872b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:55.373020 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50163d88-03a7-47ed-be51-84280b55872b None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:55.374202 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50163d88-03a7-47ed-be51-84280b55872b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:55.375637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50163d88-03a7-47ed-be51-84280b55872b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:55.377615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f797f0-ed3c-46d0-9f89-175a28336724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-88f797f0-ed3c-46d0-9f89-175a28336724 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:55.382449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:55.383438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:55.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 106/215] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:55.384837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f797f0-ed3c-46d0-9f89-175a28336724 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:55.453077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 110/216] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:11:55 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:55.454655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50163d88-03a7-47ed-be51-84280b55872b None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:55.471447 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:55.486555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:55.495091 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f797f0-ed3c-46d0-9f89-175a28336724 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:55.537314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195237a1-cbf3-45d4-8395-e23adefdba9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:55.538331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195237a1-cbf3-45d4-8395-e23adefdba9d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:55.538611 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195237a1-cbf3-45d4-8395-e23adefdba9d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:55.538881 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195237a1-cbf3-45d4-8395-e23adefdba9d None admin] PATH_INFO: `/v3/users/49527631aac643e585cc9fa743ed131f` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:55.555206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195237a1-cbf3-45d4-8395-e23adefdba9d None admin] RBAC: Authorizing `identity:get_user(user_id=49527631aac643e585cc9fa743ed131f)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:55.556739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195237a1-cbf3-45d4-8395-e23adefdba9d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:55.558765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 107/217] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:55 2022] GET /identity/v3/users/49527631aac643e585cc9fa743ed131f => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:55.564977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195237a1-cbf3-45d4-8395-e23adefdba9d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:55.575706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50163d88-03a7-47ed-be51-84280b55872b None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:55.616013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758938e-5ae5-492f-aff4-448a61ab6f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:55.616530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758938e-5ae5-492f-aff4-448a61ab6f4e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:55.616803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758938e-5ae5-492f-aff4-448a61ab6f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:55.617070 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758938e-5ae5-492f-aff4-448a61ab6f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:55.617808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758938e-5ae5-492f-aff4-448a61ab6f4e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:55.618065 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758938e-5ae5-492f-aff4-448a61ab6f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:55.619458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758938e-5ae5-492f-aff4-448a61ab6f4e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:55.629416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 111/218] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:55.635086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758938e-5ae5-492f-aff4-448a61ab6f4e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:55.684022 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195237a1-cbf3-45d4-8395-e23adefdba9d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:55.728526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:55.745077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758938e-5ae5-492f-aff4-448a61ab6f4e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:55.765843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93285ce2-5b72-4c28-a155-c2863a9e8b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:55.767479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93285ce2-5b72-4c28-a155-c2863a9e8b60 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:55.767479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93285ce2-5b72-4c28-a155-c2863a9e8b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:55.767479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93285ce2-5b72-4c28-a155-c2863a9e8b60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:55.768628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93285ce2-5b72-4c28-a155-c2863a9e8b60 None admin] RBAC: Adding query filter params (role.id=bb36a823b2ff45d29018aedf19ae7bf0, user.id=49527631aac643e585cc9fa743ed131f, scope.system=all) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:55.769017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93285ce2-5b72-4c28-a155-c2863a9e8b60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:55.771237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93285ce2-5b72-4c28-a155-c2863a9e8b60 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:55.777692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 108/219] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Oct 19 02:11:55 2022] GET /identity/v3/role_assignments?user.id=49527631aac643e585cc9fa743ed131f&scope.system=all&role.id=bb36a823b2ff45d29018aedf19ae7bf0 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:55.787584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616a25e-9a30-4f02-8666-5d1d24e4e943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:55.788238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a25e-9a30-4f02-8666-5d1d24e4e943 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:55.788517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a25e-9a30-4f02-8666-5d1d24e4e943 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:55.788785 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a25e-9a30-4f02-8666-5d1d24e4e943 None admin] PATH_INFO: `/v3/users/1939a09058d3432788297f8a2f2ea776` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:55.790007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616a25e-9a30-4f02-8666-5d1d24e4e943 None admin] RBAC: Authorizing `identity:get_user(user_id=1939a09058d3432788297f8a2f2ea776)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:55.791545 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7616a25e-9a30-4f02-8666-5d1d24e4e943 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:55.793621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 112/220] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:55 2022] GET /identity/v3/users/1939a09058d3432788297f8a2f2ea776 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:55.799545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93285ce2-5b72-4c28-a155-c2863a9e8b60 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:55.803525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:55.912217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93285ce2-5b72-4c28-a155-c2863a9e8b60 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:55.952394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a97455-54ce-4b42-abb4-3501dee758f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:55.953008 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a97455-54ce-4b42-abb4-3501dee758f4 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:55.953285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a97455-54ce-4b42-abb4-3501dee758f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:55.953556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a97455-54ce-4b42-abb4-3501dee758f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:56.023732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a97455-54ce-4b42-abb4-3501dee758f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:56.025239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a97455-54ce-4b42-abb4-3501dee758f4 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:56.027985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:56.133249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 109/221] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:11:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:11:56.138567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616a25e-9a30-4f02-8666-5d1d24e4e943 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:56.240542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616a25e-9a30-4f02-8666-5d1d24e4e943 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:56.276876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d5b4e-cba6-49aa-8e4a-eab3bbc47ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:56.277846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d5b4e-cba6-49aa-8e4a-eab3bbc47ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:56.278352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d5b4e-cba6-49aa-8e4a-eab3bbc47ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:56.278816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d5b4e-cba6-49aa-8e4a-eab3bbc47ec6 None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:56.280200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d5b4e-cba6-49aa-8e4a-eab3bbc47ec6 None admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:56.281848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d5b4e-cba6-49aa-8e4a-eab3bbc47ec6 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:56.283702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 113/222] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:11:56 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:56.294063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a97455-54ce-4b42-abb4-3501dee758f4 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:56.393775 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a97455-54ce-4b42-abb4-3501dee758f4 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:56.396162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:56.428909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de7aeef-5ea1-445a-b928-8ee79daa0a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:56.429519 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7aeef-5ea1-445a-b928-8ee79daa0a04 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:56.429858 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7aeef-5ea1-445a-b928-8ee79daa0a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:56.430164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de7aeef-5ea1-445a-b928-8ee79daa0a04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:56.430910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7aeef-5ea1-445a-b928-8ee79daa0a04 None admin] RBAC: Adding query filter params (role.id=dda0ba34c8b9442d9e12b1c8ab9a5d6c, scope.project.id=b31fe422931242de9e41c5dc9136868b, user.id=1939a09058d3432788297f8a2f2ea776) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:56.431442 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7aeef-5ea1-445a-b928-8ee79daa0a04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:56.433106 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de7aeef-5ea1-445a-b928-8ee79daa0a04 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:56.439394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 110/223] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:11:56 2022] GET /identity/v3/role_assignments?user.id=1939a09058d3432788297f8a2f2ea776&scope.project.id=b31fe422931242de9e41c5dc9136868b&role.id=dda0ba34c8b9442d9e12b1c8ab9a5d6c => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:57.778728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50e69b-8410-43d3-ae4e-9779fb47f44b None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:57.779255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50e69b-8410-43d3-ae4e-9779fb47f44b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:57.779255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50e69b-8410-43d3-ae4e-9779fb47f44b None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:57.781346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 114/224] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:57.786833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2d743-b6b4-40c0-8e16-7683662c73f9 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:57.787296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2d743-b6b4-40c0-8e16-7683662c73f9 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:57.787486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2d743-b6b4-40c0-8e16-7683662c73f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:57.806682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9ac2d743-b6b4-40c0-8e16-7683662c73f9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:57.807866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9ac2d743-b6b4-40c0-8e16-7683662c73f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:57.868364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d8a20-6823-413b-b2ab-79f9b5de471c None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:57.868643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d8a20-6823-413b-b2ab-79f9b5de471c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:57.868908 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d8a20-6823-413b-b2ab-79f9b5de471c None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:57.870477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 115/225] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:11:57.876380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06fbf0-d65b-4aae-b5bd-79cb9ab66960 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:57.876645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06fbf0-d65b-4aae-b5bd-79cb9ab66960 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:57.876876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06fbf0-d65b-4aae-b5bd-79cb9ab66960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:57.882096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 111/226] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:57.895963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-4b06fbf0-d65b-4aae-b5bd-79cb9ab66960 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:57.897133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4b06fbf0-d65b-4aae-b5bd-79cb9ab66960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:57.912342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:57.964157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 116/227] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:11:57.972687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c52a2-3374-41bb-9a44-b8b676020457 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:57.972974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c52a2-3374-41bb-9a44-b8b676020457 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:57.973244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c52a2-3374-41bb-9a44-b8b676020457 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:57.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 112/228] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:11:57.979421 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d6ae-7a96-4daf-b80d-7fb45475ad86 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:57.979707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d6ae-7a96-4daf-b80d-7fb45475ad86 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:57.979988 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387d6ae-7a96-4daf-b80d-7fb45475ad86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:57.997297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-3387d6ae-7a96-4daf-b80d-7fb45475ad86 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:11:57.998533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3387d6ae-7a96-4daf-b80d-7fb45475ad86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:11:58.062282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 117/229] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:11:58.150491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c52a2-3374-41bb-9a44-b8b676020457 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:58.229077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387d6ae-7a96-4daf-b80d-7fb45475ad86 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:58.248620 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:58.252893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c52a2-3374-41bb-9a44-b8b676020457 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:58.289392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bacca-2bdd-4c28-85b7-7dc34c085cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:58.290017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bacca-2bdd-4c28-85b7-7dc34c085cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:58.290316 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bacca-2bdd-4c28-85b7-7dc34c085cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:58.290584 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bacca-2bdd-4c28-85b7-7dc34c085cc1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:58.300368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3bacca-2bdd-4c28-85b7-7dc34c085cc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-7e3bacca-2bdd-4c28-85b7-7dc34c085cc1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:58.305128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:58.305984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:11:58.307013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 113/230] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:58.312490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bacca-2bdd-4c28-85b7-7dc34c085cc1 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:58.328230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:58.347739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387d6ae-7a96-4daf-b80d-7fb45475ad86 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:58.381702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73399501-9641-4ebf-83a9-8214e619679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:58.382353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73399501-9641-4ebf-83a9-8214e619679b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:58.382638 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73399501-9641-4ebf-83a9-8214e619679b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:58.382912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73399501-9641-4ebf-83a9-8214e619679b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:58.384011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73399501-9641-4ebf-83a9-8214e619679b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:58.385466 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73399501-9641-4ebf-83a9-8214e619679b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:58.387228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 118/231] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:11:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:58.387834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73399501-9641-4ebf-83a9-8214e619679b None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:58.415170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bacca-2bdd-4c28-85b7-7dc34c085cc1 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:58.447992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f06707-6ce9-4c36-93e8-3e8a173b730d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:58.448598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f06707-6ce9-4c36-93e8-3e8a173b730d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:58.448878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f06707-6ce9-4c36-93e8-3e8a173b730d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:58.449152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f06707-6ce9-4c36-93e8-3e8a173b730d None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:58.449811 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f06707-6ce9-4c36-93e8-3e8a173b730d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:58.450140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f06707-6ce9-4c36-93e8-3e8a173b730d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:58.452108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f06707-6ce9-4c36-93e8-3e8a173b730d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:58.460017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 114/232] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:58.461190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f06707-6ce9-4c36-93e8-3e8a173b730d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:58.483151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:58.484744 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73399501-9641-4ebf-83a9-8214e619679b None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:58.518600 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda3a8b-5483-4188-992d-14e2a2eb8aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:58.519217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda3a8b-5483-4188-992d-14e2a2eb8aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:58.519499 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda3a8b-5483-4188-992d-14e2a2eb8aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:58.519772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda3a8b-5483-4188-992d-14e2a2eb8aa6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:58.529669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda3a8b-5483-4188-992d-14e2a2eb8aa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-9fda3a8b-5483-4188-992d-14e2a2eb8aa6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:11:58.534944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:11:58.535933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:11:58.536729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 119/233] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:11:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:11:58.537376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda3a8b-5483-4188-992d-14e2a2eb8aa6 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:58.548032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:58.561278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f06707-6ce9-4c36-93e8-3e8a173b730d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:58.593599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998b6de-f17d-4e4c-8fac-20ad97d9fa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:58.594181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998b6de-f17d-4e4c-8fac-20ad97d9fa34 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:58.594465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998b6de-f17d-4e4c-8fac-20ad97d9fa34 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:58.594741 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998b6de-f17d-4e4c-8fac-20ad97d9fa34 None admin] PATH_INFO: `/v3/groups` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:58.595377 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998b6de-f17d-4e4c-8fac-20ad97d9fa34 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:58.596807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998b6de-f17d-4e4c-8fac-20ad97d9fa34 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:58.634550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda3a8b-5483-4188-992d-14e2a2eb8aa6 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:58.665770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f730f-a599-4226-add8-4de565e10400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:58.666414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f730f-a599-4226-add8-4de565e10400 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:58.666695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f730f-a599-4226-add8-4de565e10400 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:58.666967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f730f-a599-4226-add8-4de565e10400 None admin] PATH_INFO: `/v3/users/4296da26da354e9b9b5f136c0b5c1f36` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:58.680875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f730f-a599-4226-add8-4de565e10400 None admin] RBAC: Authorizing `identity:get_user(user_id=4296da26da354e9b9b5f136c0b5c1f36)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:58.682349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f730f-a599-4226-add8-4de565e10400 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:58.683918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 120/234] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:58 2022] GET /identity/v3/users/4296da26da354e9b9b5f136c0b5c1f36 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:58.684673 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f730f-a599-4226-add8-4de565e10400 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:58.696162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 115/235] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Oct 19 02:11:58 2022] POST /identity/v3/groups => generated 229 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:11:58.696979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998b6de-f17d-4e4c-8fac-20ad97d9fa34 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:58.768583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:58.798721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:58.810197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f730f-a599-4226-add8-4de565e10400 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:58.819015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998b6de-f17d-4e4c-8fac-20ad97d9fa34 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:58.849967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d69191-51b8-49ca-9ace-e1891d9c79ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:58.850545 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d69191-51b8-49ca-9ace-e1891d9c79ce None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:58.850824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d69191-51b8-49ca-9ace-e1891d9c79ce None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:58.851096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d69191-51b8-49ca-9ace-e1891d9c79ce None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:58.851768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d69191-51b8-49ca-9ace-e1891d9c79ce None admin] RBAC: Adding query filter params (name=member) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:58.852100 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d69191-51b8-49ca-9ace-e1891d9c79ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:58.853512 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d69191-51b8-49ca-9ace-e1891d9c79ce None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:58.858516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc3b2d-d854-49fd-97ef-946242a272c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:58.859404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc3b2d-d854-49fd-97ef-946242a272c7 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:58.859873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc3b2d-d854-49fd-97ef-946242a272c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:58.860147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc3b2d-d854-49fd-97ef-946242a272c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:58.860889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc3b2d-d854-49fd-97ef-946242a272c7 None admin] RBAC: Adding query filter params (role.id=dda0ba34c8b9442d9e12b1c8ab9a5d6c, user.id=4296da26da354e9b9b5f136c0b5c1f36, scope.system=all) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:11:58.861194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc3b2d-d854-49fd-97ef-946242a272c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:58.862759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc3b2d-d854-49fd-97ef-946242a272c7 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:58.863288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 121/236] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:11:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:58.868654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 116/237] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Oct 19 02:11:58 2022] GET /identity/v3/role_assignments?user.id=4296da26da354e9b9b5f136c0b5c1f36&scope.system=all&role.id=dda0ba34c8b9442d9e12b1c8ab9a5d6c => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:11:58.868870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d69191-51b8-49ca-9ace-e1891d9c79ce None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:58.984351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d69191-51b8-49ca-9ace-e1891d9c79ce None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:59.014034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:11:59.022455 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03ba17-145a-4fbc-a362-03eb65ffdec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:59.023098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ba17-145a-4fbc-a362-03eb65ffdec8 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:59.023373 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ba17-145a-4fbc-a362-03eb65ffdec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:59.023641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03ba17-145a-4fbc-a362-03eb65ffdec8 None admin] PATH_INFO: `/v3/users/49527631aac643e585cc9fa743ed131f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:59.024868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03ba17-145a-4fbc-a362-03eb65ffdec8 None admin] RBAC: Authorizing `identity:get_user(user_id=49527631aac643e585cc9fa743ed131f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:59.026424 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03ba17-145a-4fbc-a362-03eb65ffdec8 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:59.028303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 122/238] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:11:58 2022] GET /identity/v3/users/49527631aac643e585cc9fa743ed131f => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:11:59.034794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc3b2d-d854-49fd-97ef-946242a272c7 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:11:59.147360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc3b2d-d854-49fd-97ef-946242a272c7 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:11:59.181725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda410e6-ead3-41b0-8945-bf5e7a441ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:11:59.182404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda410e6-ead3-41b0-8945-bf5e7a441ff2 None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:11:59.182685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda410e6-ead3-41b0-8945-bf5e7a441ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:11:59.182959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda410e6-ead3-41b0-8945-bf5e7a441ff2 None admin] PATH_INFO: `/v3/system/users/49527631aac643e585cc9fa743ed131f/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:11:59.184372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda410e6-ead3-41b0-8945-bf5e7a441ff2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=49527631aac643e585cc9fa743ed131f, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:11:59.185880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda410e6-ead3-41b0-8945-bf5e7a441ff2 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:11:59.276634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 117/239] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Oct 19 02:11:59 2022] PUT /identity/v3/system/users/49527631aac643e585cc9fa743ed131f/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:00.900344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35234d58-a3d7-4afd-9e99-535f51635cf8 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:00.900872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35234d58-a3d7-4afd-9e99-535f51635cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:00.900872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35234d58-a3d7-4afd-9e99-535f51635cf8 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:00.902540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 123/240] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:00.908224 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbf446-9963-44cc-a33a-36065c52aace None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:00.908723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbf446-9963-44cc-a33a-36065c52aace None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:00.908794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbf446-9963-44cc-a33a-36065c52aace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:00.929344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-efbbf446-9963-44cc-a33a-36065c52aace None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:00.930579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-efbbf446-9963-44cc-a33a-36065c52aace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:00.932725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:00.997647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 118/241] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:01.016540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df618453-49df-4358-a5f0-9c5dd27600af None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:01.016822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df618453-49df-4358-a5f0-9c5dd27600af None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:01.017092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df618453-49df-4358-a5f0-9c5dd27600af None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:01.018598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 124/242] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:01.023789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85257894-05c7-4af2-8d4e-3583e74071f4 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:01.024314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85257894-05c7-4af2-8d4e-3583e74071f4 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:01.024789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85257894-05c7-4af2-8d4e-3583e74071f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:01.043313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-85257894-05c7-4af2-8d4e-3583e74071f4 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:01.044769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-85257894-05c7-4af2-8d4e-3583e74071f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:01.116142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 119/243] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:01.271987 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df618453-49df-4358-a5f0-9c5dd27600af None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:01.378839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df618453-49df-4358-a5f0-9c5dd27600af None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:01.386274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85257894-05c7-4af2-8d4e-3583e74071f4 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:01.415437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc44ecc-4a1d-40f5-9558-56da669d0142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:01.416070 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc44ecc-4a1d-40f5-9558-56da669d0142 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:01.416348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc44ecc-4a1d-40f5-9558-56da669d0142 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:01.416620 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc44ecc-4a1d-40f5-9558-56da669d0142 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:01.417732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc44ecc-4a1d-40f5-9558-56da669d0142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:01.419296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc44ecc-4a1d-40f5-9558-56da669d0142 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:01.421021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 125/244] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:01.422144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482cb7-f371-463d-94ce-c2d9bd45d74e None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:01.422425 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482cb7-f371-463d-94ce-c2d9bd45d74e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:01.422695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29482cb7-f371-463d-94ce-c2d9bd45d74e None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:01.424017 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 126/245] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:01.426029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29482cb7-f371-463d-94ce-c2d9bd45d74e None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:01.453719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:01.484803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85257894-05c7-4af2-8d4e-3583e74071f4 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:01.488141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:01.519007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aabdbc-4799-49d0-90f1-440d29f0eda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:01.519635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aabdbc-4799-49d0-90f1-440d29f0eda1 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:01.519913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aabdbc-4799-49d0-90f1-440d29f0eda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:01.520184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aabdbc-4799-49d0-90f1-440d29f0eda1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:01.526309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29482cb7-f371-463d-94ce-c2d9bd45d74e None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:01.530187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aabdbc-4799-49d0-90f1-440d29f0eda1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-23aabdbc-4799-49d0-90f1-440d29f0eda1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:01.538864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:01.539728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 120/246] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:01.541929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191f34-389e-4534-b9d1-ad03ceddb666 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:01.543394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191f34-389e-4534-b9d1-ad03ceddb666 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:01.543394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191f34-389e-4534-b9d1-ad03ceddb666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:01.560072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9c191f34-389e-4534-b9d1-ad03ceddb666 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:01.561835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9c191f34-389e-4534-b9d1-ad03ceddb666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:01.565398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557420e8-1e75-4a7d-944a-0cac55b7682d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:01.566000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557420e8-1e75-4a7d-944a-0cac55b7682d None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:01.566303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557420e8-1e75-4a7d-944a-0cac55b7682d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:01.566570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557420e8-1e75-4a7d-944a-0cac55b7682d None admin] PATH_INFO: `/v3/groups` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:01.567192 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557420e8-1e75-4a7d-944a-0cac55b7682d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:01.568621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557420e8-1e75-4a7d-944a-0cac55b7682d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:01.626386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 121/247] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:01.627200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c191f34-389e-4534-b9d1-ad03ceddb666 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:01.681134 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:01.682571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 127/248] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Oct 19 02:12:01 2022] POST /identity/v3/groups => generated 226 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:12:01.729976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c191f34-389e-4534-b9d1-ad03ceddb666 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:01.769360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d551a-f6f5-4266-9750-c6a35fcc5cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:01.770487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d551a-f6f5-4266-9750-c6a35fcc5cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:01.770918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d551a-f6f5-4266-9750-c6a35fcc5cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:01.771337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d551a-f6f5-4266-9750-c6a35fcc5cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:01.772259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d551a-f6f5-4266-9750-c6a35fcc5cc0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:01.772708 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d551a-f6f5-4266-9750-c6a35fcc5cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:01.775210 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d551a-f6f5-4266-9750-c6a35fcc5cc0 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:01.784115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 122/249] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:12:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:01.789586 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557420e8-1e75-4a7d-944a-0cac55b7682d None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:01.900769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d551a-f6f5-4266-9750-c6a35fcc5cc0 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:01.901180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557420e8-1e75-4a7d-944a-0cac55b7682d None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:01.930030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:01.940572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa92e0-0785-4577-833d-d2f19b3538f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:01.941206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa92e0-0785-4577-833d-d2f19b3538f0 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:01.941479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa92e0-0785-4577-833d-d2f19b3538f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:01.941753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa92e0-0785-4577-833d-d2f19b3538f0 None admin] PATH_INFO: `/v3/users/4296da26da354e9b9b5f136c0b5c1f36` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:01.942924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa92e0-0785-4577-833d-d2f19b3538f0 None admin] RBAC: Authorizing `identity:get_user(user_id=4296da26da354e9b9b5f136c0b5c1f36)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:01.945637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa92e0-0785-4577-833d-d2f19b3538f0 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:01.946113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 128/250] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:12:01 2022] GET /identity/v3/users/4296da26da354e9b9b5f136c0b5c1f36 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:01.951492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa92e0-0785-4577-833d-d2f19b3538f0 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:02.034081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d551a-f6f5-4266-9750-c6a35fcc5cc0 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:02.064787 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:02.091154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb90916-d5b2-4a21-885a-364866bfae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:02.092866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb90916-d5b2-4a21-885a-364866bfae6b None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:02.093160 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb90916-d5b2-4a21-885a-364866bfae6b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:02.093451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb90916-d5b2-4a21-885a-364866bfae6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:02.095184 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa92e0-0785-4577-833d-d2f19b3538f0 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:02.118813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb90916-d5b2-4a21-885a-364866bfae6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-0cb90916-d5b2-4a21-885a-364866bfae6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:02.126689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:02.127818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:02.128607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 123/251] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:02.134345 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb90916-d5b2-4a21-885a-364866bfae6b None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:02.147573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80d49d-e697-4275-bb24-8daf9a40bb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:02.148240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80d49d-e697-4275-bb24-8daf9a40bb78 None admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:02.148522 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80d49d-e697-4275-bb24-8daf9a40bb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:02.148897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80d49d-e697-4275-bb24-8daf9a40bb78 None admin] PATH_INFO: `/v3/system/users/4296da26da354e9b9b5f136c0b5c1f36/roles/dda0ba34c8b9442d9e12b1c8ab9a5d6c` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:02.150633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80d49d-e697-4275-bb24-8daf9a40bb78 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4296da26da354e9b9b5f136c0b5c1f36, role_id=dda0ba34c8b9442d9e12b1c8ab9a5d6c)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:02.152196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80d49d-e697-4275-bb24-8daf9a40bb78 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:02.250162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb90916-d5b2-4a21-885a-364866bfae6b None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:02.288027 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d2cd6-fa2c-473d-97c6-6faebc6eed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:02.288688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d2cd6-fa2c-473d-97c6-6faebc6eed24 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:02.288971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d2cd6-fa2c-473d-97c6-6faebc6eed24 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:02.289241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d2cd6-fa2c-473d-97c6-6faebc6eed24 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:02.289907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d2cd6-fa2c-473d-97c6-6faebc6eed24 None admin] RBAC: Adding query filter params (name=member) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:02.290237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d2cd6-fa2c-473d-97c6-6faebc6eed24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:02.291605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d2cd6-fa2c-473d-97c6-6faebc6eed24 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:02.302376 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 124/252] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:12:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:02.306868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 129/253] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Oct 19 02:12:01 2022] PUT /identity/v3/system/users/4296da26da354e9b9b5f136c0b5c1f36/roles/dda0ba34c8b9442d9e12b1c8ab9a5d6c => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:12:02.309566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d2cd6-fa2c-473d-97c6-6faebc6eed24 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:02.362985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:02.431717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d2cd6-fa2c-473d-97c6-6faebc6eed24 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:02.473927 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781879b7-98f2-4760-a595-c9923d260e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:02.474668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781879b7-98f2-4760-a595-c9923d260e4f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:02.474947 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781879b7-98f2-4760-a595-c9923d260e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:02.475217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781879b7-98f2-4760-a595-c9923d260e4f None admin] PATH_INFO: `/v3/users/49527631aac643e585cc9fa743ed131f` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:02.476521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781879b7-98f2-4760-a595-c9923d260e4f None admin] RBAC: Authorizing `identity:get_user(user_id=49527631aac643e585cc9fa743ed131f)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:02.478132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781879b7-98f2-4760-a595-c9923d260e4f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:02.480099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 125/254] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:12:02 2022] GET /identity/v3/users/49527631aac643e585cc9fa743ed131f => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:02.486507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80d49d-e697-4275-bb24-8daf9a40bb78 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:02.551922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:02.599393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80d49d-e697-4275-bb24-8daf9a40bb78 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:02.637942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697bc56-c3b6-4e3c-9184-dc1f18bc534d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:02.638536 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697bc56-c3b6-4e3c-9184-dc1f18bc534d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:02.639377 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697bc56-c3b6-4e3c-9184-dc1f18bc534d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:02.639669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697bc56-c3b6-4e3c-9184-dc1f18bc534d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:02.640396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697bc56-c3b6-4e3c-9184-dc1f18bc534d None admin] RBAC: Adding query filter params (role.id=bb36a823b2ff45d29018aedf19ae7bf0, user.id=49527631aac643e585cc9fa743ed131f, scope.system=all) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:02.640703 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697bc56-c3b6-4e3c-9184-dc1f18bc534d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:02.642172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697bc56-c3b6-4e3c-9184-dc1f18bc534d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:02.654353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 130/255] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Oct 19 02:12:02 2022] GET /identity/v3/role_assignments?user.id=49527631aac643e585cc9fa743ed131f&scope.system=all&role.id=bb36a823b2ff45d29018aedf19ae7bf0 => generated 510 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:04.151898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb808748-f25b-4be5-8dd6-34e696af1870 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.152886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb808748-f25b-4be5-8dd6-34e696af1870 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.153364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb808748-f25b-4be5-8dd6-34e696af1870 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.155511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 126/256] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:04.172149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed7e14-89fb-42af-861b-f80c2ccafe88 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.173063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed7e14-89fb-42af-861b-f80c2ccafe88 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.173536 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed7e14-89fb-42af-861b-f80c2ccafe88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.199816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-9fed7e14-89fb-42af-861b-f80c2ccafe88 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:04.201503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9fed7e14-89fb-42af-861b-f80c2ccafe88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:04.284244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 131/257] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 19 02:12:04.336058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daccfd-421b-41e3-9a45-c226922974c3 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.336697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daccfd-421b-41e3-9a45-c226922974c3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.337412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96daccfd-421b-41e3-9a45-c226922974c3 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.338980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 127/258] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:04.346522 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f56616-a212-4776-a4c8-047b791709de None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.347076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f56616-a212-4776-a4c8-047b791709de None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.347540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f56616-a212-4776-a4c8-047b791709de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.367600 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e0f56616-a212-4776-a4c8-047b791709de None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:04.369055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e0f56616-a212-4776-a4c8-047b791709de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:04.377341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 132/259] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 19 02:12:04.657598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daccfd-421b-41e3-9a45-c226922974c3 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:04.671854 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96daccfd-421b-41e3-9a45-c226922974c3 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:04.676628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb4847-9d7a-43cf-b50a-44e6fce7bcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:04.677498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb4847-9d7a-43cf-b50a-44e6fce7bcc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.677968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb4847-9d7a-43cf-b50a-44e6fce7bcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.678472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb4847-9d7a-43cf-b50a-44e6fce7bcc9 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.690867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb4847-9d7a-43cf-b50a-44e6fce7bcc9 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-efbb4847-9d7a-43cf-b50a-44e6fce7bcc9 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:04.696198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:04.697037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:12:04.697788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:04.700333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 128/260] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 19 02:12:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:04.706258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f56616-a212-4776-a4c8-047b791709de None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:04.716022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f56616-a212-4776-a4c8-047b791709de None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:04.720776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae80e5d-6ba8-40d4-b0f4-800e70bbb14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:04.721602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae80e5d-6ba8-40d4-b0f4-800e70bbb14a admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.722106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae80e5d-6ba8-40d4-b0f4-800e70bbb14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.722592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae80e5d-6ba8-40d4-b0f4-800e70bbb14a admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.723445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae80e5d-6ba8-40d4-b0f4-800e70bbb14a admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:04.723933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae80e5d-6ba8-40d4-b0f4-800e70bbb14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:04.725946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: WARNING py.warnings [None req-2ae80e5d-6ba8-40d4-b0f4-800e70bbb14a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:12:04.725946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: warnings.warn(msg) Oct 19 02:12:04.725946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: Oct 19 02:12:04.727244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae80e5d-6ba8-40d4-b0f4-800e70bbb14a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:04.737159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 133/261] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Oct 19 02:12:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:04.743020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb4847-9d7a-43cf-b50a-44e6fce7bcc9 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:04.753631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb4847-9d7a-43cf-b50a-44e6fce7bcc9 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:04.758642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631da16-b853-48f2-86f8-5fff78dfa848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:04.759527 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631da16-b853-48f2-86f8-5fff78dfa848 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.760009 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631da16-b853-48f2-86f8-5fff78dfa848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.760469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631da16-b853-48f2-86f8-5fff78dfa848 admin admin] PATH_INFO: `/v3/groups/22cc5bce0f1f40448cd8041a36289b21` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.777160 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631da16-b853-48f2-86f8-5fff78dfa848 admin admin] RBAC: Authorizing `identity:get_group(group_id=22cc5bce0f1f40448cd8041a36289b21)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:04.779328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: WARNING py.warnings [None req-7631da16-b853-48f2-86f8-5fff78dfa848 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:12:04.779328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: warnings.warn(msg) Oct 19 02:12:04.779328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: Oct 19 02:12:04.781528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631da16-b853-48f2-86f8-5fff78dfa848 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:04.783481 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 129/262] 10.222.0.20 () {56 vars in 1201 bytes} [Wed Oct 19 02:12:04 2022] GET /identity/v3/groups/22cc5bce0f1f40448cd8041a36289b21 => generated 226 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:04.789240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae80e5d-6ba8-40d4-b0f4-800e70bbb14a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:04.801158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae80e5d-6ba8-40d4-b0f4-800e70bbb14a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:04.803033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631da16-b853-48f2-86f8-5fff78dfa848 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:04.811160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a485895-7d70-4dad-bf2a-2c13e6042f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:04.811814 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a485895-7d70-4dad-bf2a-2c13e6042f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.812116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a485895-7d70-4dad-bf2a-2c13e6042f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.812374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a485895-7d70-4dad-bf2a-2c13e6042f0b admin admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.814101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a485895-7d70-4dad-bf2a-2c13e6042f0b admin admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:04.816186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a485895-7d70-4dad-bf2a-2c13e6042f0b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:04.817996 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 134/263] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 19 02:12:04 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:04.824379 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631da16-b853-48f2-86f8-5fff78dfa848 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:04.828738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a485895-7d70-4dad-bf2a-2c13e6042f0b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:04.832823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9055e8a-c90f-47eb-bed5-b7c776b0ca58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:04.833761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9055e8a-c90f-47eb-bed5-b7c776b0ca58 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.834494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9055e8a-c90f-47eb-bed5-b7c776b0ca58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.834999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9055e8a-c90f-47eb-bed5-b7c776b0ca58 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.848010 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a485895-7d70-4dad-bf2a-2c13e6042f0b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:04.857271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1018a59-227b-4997-b93d-c6fe39d8a880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:04.859124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1018a59-227b-4997-b93d-c6fe39d8a880 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.859849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1018a59-227b-4997-b93d-c6fe39d8a880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.860568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9055e8a-c90f-47eb-bed5-b7c776b0ca58 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:04.861073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1018a59-227b-4997-b93d-c6fe39d8a880 admin admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf/groups/22cc5bce0f1f40448cd8041a36289b21/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.865913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1018a59-227b-4997-b93d-c6fe39d8a880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce320a8ccf64f479c1dae1117865baf, group_id=22cc5bce0f1f40448cd8041a36289b21, role_id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:04.868183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: WARNING py.warnings [None req-e1018a59-227b-4997-b93d-c6fe39d8a880 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:12:04.868183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: warnings.warn(msg) Oct 19 02:12:04.868183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: Oct 19 02:12:04.872655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1018a59-227b-4997-b93d-c6fe39d8a880 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-c9055e8a-c90f-47eb-bed5-b7c776b0ca58 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:04.872889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:04.873772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:12:04.874592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:04.876971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 130/264] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 19 02:12:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 74 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:04.882809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9055e8a-c90f-47eb-bed5-b7c776b0ca58 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:04.894827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9055e8a-c90f-47eb-bed5-b7c776b0ca58 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:04.900987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e2d6-eb92-4a25-841f-07e3f482219e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:04.900987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e2d6-eb92-4a25-841f-07e3f482219e admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.900987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e2d6-eb92-4a25-841f-07e3f482219e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.901465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4e2d6-eb92-4a25-841f-07e3f482219e admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.902411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4e2d6-eb92-4a25-841f-07e3f482219e admin admin] RBAC: Adding query filter params (name=member) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:04.902964 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4e2d6-eb92-4a25-841f-07e3f482219e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:04.904260 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: WARNING py.warnings [None req-1aa4e2d6-eb92-4a25-841f-07e3f482219e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:12:04.904260 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: warnings.warn(msg) Oct 19 02:12:04.904260 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: Oct 19 02:12:04.906109 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4e2d6-eb92-4a25-841f-07e3f482219e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:04.916095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 131/265] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:12:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:04.924058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e2d6-eb92-4a25-841f-07e3f482219e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:04.934801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4e2d6-eb92-4a25-841f-07e3f482219e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:04.940351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f055f-a6b0-4158-9337-7fa6dadacd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:04.941295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f055f-a6b0-4158-9337-7fa6dadacd97 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:04.941797 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f055f-a6b0-4158-9337-7fa6dadacd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:04.942300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f055f-a6b0-4158-9337-7fa6dadacd97 admin admin] PATH_INFO: `/v3/groups/22cc5bce0f1f40448cd8041a36289b21` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:04.943646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f055f-a6b0-4158-9337-7fa6dadacd97 admin admin] RBAC: Authorizing `identity:get_group(group_id=22cc5bce0f1f40448cd8041a36289b21)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:04.953324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 135/266] 10.222.0.20 () {58 vars in 1463 bytes} [Wed Oct 19 02:12:04 2022] PUT /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf/groups/22cc5bce0f1f40448cd8041a36289b21/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:04.954668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f055f-a6b0-4158-9337-7fa6dadacd97 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:04.957949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 132/267] 10.222.0.20 () {56 vars in 1201 bytes} [Wed Oct 19 02:12:04 2022] GET /identity/v3/groups/22cc5bce0f1f40448cd8041a36289b21 => generated 226 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:04.962686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1018a59-227b-4997-b93d-c6fe39d8a880 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:05.027796 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:05.064992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1018a59-227b-4997-b93d-c6fe39d8a880 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:05.072682 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9ca90-a43b-440f-bede-503249bc0c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:05.073356 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9ca90-a43b-440f-bede-503249bc0c8e admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:05.073639 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9ca90-a43b-440f-bede-503249bc0c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:05.073916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9ca90-a43b-440f-bede-503249bc0c8e admin admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:05.075186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9ca90-a43b-440f-bede-503249bc0c8e admin admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:05.077388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9ca90-a43b-440f-bede-503249bc0c8e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:05.079487 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 136/268] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 19 02:12:04 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:05.085452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f055f-a6b0-4158-9337-7fa6dadacd97 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:05.105073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f055f-a6b0-4158-9337-7fa6dadacd97 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:05.114736 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369d501-c214-4f4f-8415-26a0116434c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:05.115463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369d501-c214-4f4f-8415-26a0116434c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:05.115739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369d501-c214-4f4f-8415-26a0116434c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:05.116009 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369d501-c214-4f4f-8415-26a0116434c4 admin admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf/groups/22cc5bce0f1f40448cd8041a36289b21/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:05.124281 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369d501-c214-4f4f-8415-26a0116434c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce320a8ccf64f479c1dae1117865baf, group_id=22cc5bce0f1f40448cd8041a36289b21, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:05.125381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: WARNING py.warnings [None req-4369d501-c214-4f4f-8415-26a0116434c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:12:05.125381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: warnings.warn(msg) Oct 19 02:12:05.125381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: Oct 19 02:12:05.127431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369d501-c214-4f4f-8415-26a0116434c4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:05.204173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 133/269] 10.222.0.20 () {58 vars in 1463 bytes} [Wed Oct 19 02:12:05 2022] PUT /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf/groups/22cc5bce0f1f40448cd8041a36289b21/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:05.355929 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98250d81-383e-4a9e-976b-408d7f05fe81 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:05.356431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98250d81-383e-4a9e-976b-408d7f05fe81 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:05.356535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98250d81-383e-4a9e-976b-408d7f05fe81 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:05.358175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 137/270] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:05.378948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132abfe4-7467-45ec-b146-dc4446bee492 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:05.379264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132abfe4-7467-45ec-b146-dc4446bee492 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:05.379496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132abfe4-7467-45ec-b146-dc4446bee492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:05.405662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-132abfe4-7467-45ec-b146-dc4446bee492 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:05.407071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-132abfe4-7467-45ec-b146-dc4446bee492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:05.470353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:05.490002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 134/271] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:05.789426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7480d2-6341-4f00-8834-226eeb2c3edb None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:05.789815 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7480d2-6341-4f00-8834-226eeb2c3edb None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:05.790011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7480d2-6341-4f00-8834-226eeb2c3edb None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:05.791588 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 138/272] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:05.803439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1179c3-3109-4296-a9de-3550cc49a3e0 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:05.803724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1179c3-3109-4296-a9de-3550cc49a3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:05.803996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1179c3-3109-4296-a9de-3550cc49a3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:05.825129 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ee1179c3-3109-4296-a9de-3550cc49a3e0 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:05.826408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ee1179c3-3109-4296-a9de-3550cc49a3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:05.881681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7480d2-6341-4f00-8834-226eeb2c3edb None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:05.931418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 135/273] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 19 02:12:06.014605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7480d2-6341-4f00-8834-226eeb2c3edb None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:06.050166 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78ac7a-a9ff-420e-b8ae-98cf6267b319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:06.050732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78ac7a-a9ff-420e-b8ae-98cf6267b319 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:06.051009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78ac7a-a9ff-420e-b8ae-98cf6267b319 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:06.051314 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78ac7a-a9ff-420e-b8ae-98cf6267b319 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:06.063256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd78ac7a-a9ff-420e-b8ae-98cf6267b319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-fd78ac7a-a9ff-420e-b8ae-98cf6267b319 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:06.068355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:06.069218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 19 02:12:06.070014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:06.078064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 139/274] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:06.078621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1179c3-3109-4296-a9de-3550cc49a3e0 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:06.223567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1179c3-3109-4296-a9de-3550cc49a3e0 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:06.254450 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:06.267640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b25479-4770-4882-a0b9-7d058504c762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:06.268269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25479-4770-4882-a0b9-7d058504c762 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:06.268547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25479-4770-4882-a0b9-7d058504c762 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:06.271959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b25479-4770-4882-a0b9-7d058504c762 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:06.271959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b25479-4770-4882-a0b9-7d058504c762 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:06.271959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b25479-4770-4882-a0b9-7d058504c762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:06.271959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b25479-4770-4882-a0b9-7d058504c762 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:06.279929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 136/275] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:12:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:06.285627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78ac7a-a9ff-420e-b8ae-98cf6267b319 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:06.348725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:06.448011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78ac7a-a9ff-420e-b8ae-98cf6267b319 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:06.485572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57add0db-6c95-44ed-b87f-135f8dc78e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:06.486566 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57add0db-6c95-44ed-b87f-135f8dc78e13 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:06.487067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57add0db-6c95-44ed-b87f-135f8dc78e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:06.487552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57add0db-6c95-44ed-b87f-135f8dc78e13 None admin] PATH_INFO: `/v3/users/4296da26da354e9b9b5f136c0b5c1f36` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:06.488901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57add0db-6c95-44ed-b87f-135f8dc78e13 None admin] RBAC: Authorizing `identity:get_user(user_id=4296da26da354e9b9b5f136c0b5c1f36)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:06.490612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57add0db-6c95-44ed-b87f-135f8dc78e13 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:06.492517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 140/276] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:12:06 2022] GET /identity/v3/users/4296da26da354e9b9b5f136c0b5c1f36 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:06.499254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b25479-4770-4882-a0b9-7d058504c762 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:06.600758 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57add0db-6c95-44ed-b87f-135f8dc78e13 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:06.620258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57add0db-6c95-44ed-b87f-135f8dc78e13 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:06.633226 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa141e30-8b97-4bbe-8ee3-fe143f5868fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:06.634212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa141e30-8b97-4bbe-8ee3-fe143f5868fb admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:06.634712 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa141e30-8b97-4bbe-8ee3-fe143f5868fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:06.635195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa141e30-8b97-4bbe-8ee3-fe143f5868fb admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:06.636211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b25479-4770-4882-a0b9-7d058504c762 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:06.657091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa141e30-8b97-4bbe-8ee3-fe143f5868fb admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-aa141e30-8b97-4bbe-8ee3-fe143f5868fb admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:06.667913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:06.668848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:12:06.669618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:06.672313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 141/277] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 19 02:12:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 72 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:06.680080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa141e30-8b97-4bbe-8ee3-fe143f5868fb admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:06.697118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa141e30-8b97-4bbe-8ee3-fe143f5868fb admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:06.701813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b612a4e-89d4-43cb-b22d-d529d1bb5469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:06.702498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b612a4e-89d4-43cb-b22d-d529d1bb5469 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:06.702750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b612a4e-89d4-43cb-b22d-d529d1bb5469 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:06.703033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b612a4e-89d4-43cb-b22d-d529d1bb5469 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:06.703761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b612a4e-89d4-43cb-b22d-d529d1bb5469 None admin] RBAC: Adding query filter params (role.id=dda0ba34c8b9442d9e12b1c8ab9a5d6c, user.id=4296da26da354e9b9b5f136c0b5c1f36, scope.system=all) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:06.704083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b612a4e-89d4-43cb-b22d-d529d1bb5469 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:06.705502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b612a4e-89d4-43cb-b22d-d529d1bb5469 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:06.705979 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c7f80-5131-4060-93f9-795a5791088f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:06.706931 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c7f80-5131-4060-93f9-795a5791088f admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:06.707405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c7f80-5131-4060-93f9-795a5791088f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:06.707879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c7f80-5131-4060-93f9-795a5791088f admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:06.708812 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c7f80-5131-4060-93f9-795a5791088f admin admin] RBAC: Adding query filter params (name=member) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:06.709305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c7f80-5131-4060-93f9-795a5791088f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:06.711204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c7f80-5131-4060-93f9-795a5791088f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:06.729126 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 137/278] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Oct 19 02:12:06 2022] GET /identity/v3/role_assignments?user.id=4296da26da354e9b9b5f136c0b5c1f36&scope.system=all&role.id=dda0ba34c8b9442d9e12b1c8ab9a5d6c => generated 510 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:06.729609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 142/279] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:12:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:06.734868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b612a4e-89d4-43cb-b22d-d529d1bb5469 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:06.754432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b612a4e-89d4-43cb-b22d-d529d1bb5469 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:06.765060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e484c21-9562-4d9b-9cc0-bc9099f43bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:06.765719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e484c21-9562-4d9b-9cc0-bc9099f43bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:06.766000 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e484c21-9562-4d9b-9cc0-bc9099f43bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:06.766304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e484c21-9562-4d9b-9cc0-bc9099f43bb4 admin admin] PATH_INFO: `/v3/groups/22cc5bce0f1f40448cd8041a36289b21` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:06.767451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e484c21-9562-4d9b-9cc0-bc9099f43bb4 admin admin] RBAC: Authorizing `identity:get_group(group_id=22cc5bce0f1f40448cd8041a36289b21)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:06.769376 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e484c21-9562-4d9b-9cc0-bc9099f43bb4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:06.772324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 138/280] 10.222.0.20 () {56 vars in 1201 bytes} [Wed Oct 19 02:12:06 2022] GET /identity/v3/groups/22cc5bce0f1f40448cd8041a36289b21 => generated 226 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:06.777004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c7f80-5131-4060-93f9-795a5791088f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:06.792079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c7f80-5131-4060-93f9-795a5791088f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:06.801243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09aa182-4fe9-431e-bc1b-7049df6702f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:06.802286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09aa182-4fe9-431e-bc1b-7049df6702f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:06.802765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09aa182-4fe9-431e-bc1b-7049df6702f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:06.803232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09aa182-4fe9-431e-bc1b-7049df6702f1 admin admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:06.807254 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09aa182-4fe9-431e-bc1b-7049df6702f1 admin admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:06.809692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09aa182-4fe9-431e-bc1b-7049df6702f1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:06.811795 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 143/281] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 19 02:12:06 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:06.817975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e484c21-9562-4d9b-9cc0-bc9099f43bb4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:06.832234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e484c21-9562-4d9b-9cc0-bc9099f43bb4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:06.842678 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07fedd-0ffb-4cc5-8917-72450cf468eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:06.843360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07fedd-0ffb-4cc5-8917-72450cf468eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:06.843644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07fedd-0ffb-4cc5-8917-72450cf468eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:06.843920 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07fedd-0ffb-4cc5-8917-72450cf468eb admin admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b/groups/22cc5bce0f1f40448cd8041a36289b21/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:06.851831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07fedd-0ffb-4cc5-8917-72450cf468eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31fe422931242de9e41c5dc9136868b, group_id=22cc5bce0f1f40448cd8041a36289b21, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:06.854511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07fedd-0ffb-4cc5-8917-72450cf468eb admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:06.978119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cc73a-0f05-4634-81ad-5fb2141b9964 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:06.978425 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cc73a-0f05-4634-81ad-5fb2141b9964 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:06.978672 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cc73a-0f05-4634-81ad-5fb2141b9964 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:06.980012 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 144/282] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:06.985808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e1385-47fe-404b-9317-7bfd1c392ba7 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:06.986128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e1385-47fe-404b-9317-7bfd1c392ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:06.986403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e1385-47fe-404b-9317-7bfd1c392ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:06.989836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 139/283] 10.222.0.20 () {58 vars in 1463 bytes} [Wed Oct 19 02:12:06 2022] PUT /identity/v3/projects/b31fe422931242de9e41c5dc9136868b/groups/22cc5bce0f1f40448cd8041a36289b21/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:07.008417 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-cf4e1385-47fe-404b-9317-7bfd1c392ba7 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:07.009708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cf4e1385-47fe-404b-9317-7bfd1c392ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:07.023397 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7673048-c62a-46e1-b051-68a1717eb966 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.023684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7673048-c62a-46e1-b051-68a1717eb966 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.023960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7673048-c62a-46e1-b051-68a1717eb966 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.026148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 140/284] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:12:07.033071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a77efe-47c1-402e-bcaf-770bc8938466 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.033980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a77efe-47c1-402e-bcaf-770bc8938466 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.034457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a77efe-47c1-402e-bcaf-770bc8938466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.071141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-f2a77efe-47c1-402e-bcaf-770bc8938466 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:07.072747 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f2a77efe-47c1-402e-bcaf-770bc8938466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:07.107229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 145/285] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 19 02:12:07.128590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:07.163112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 141/286] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 19 02:12:07.445520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e1385-47fe-404b-9317-7bfd1c392ba7 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:07.455457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e1385-47fe-404b-9317-7bfd1c392ba7 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:07.461677 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfee7d4-29e6-41a8-ae4a-51dcb343487f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:07.462612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfee7d4-29e6-41a8-ae4a-51dcb343487f admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.462891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfee7d4-29e6-41a8-ae4a-51dcb343487f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.463164 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfee7d4-29e6-41a8-ae4a-51dcb343487f admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.473813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfee7d4-29e6-41a8-ae4a-51dcb343487f admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-fcfee7d4-29e6-41a8-ae4a-51dcb343487f admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:07.478661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:07.479521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:07.480332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:07.484989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 146/287] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 19 02:12:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:07.485899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a77efe-47c1-402e-bcaf-770bc8938466 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:07.497049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a77efe-47c1-402e-bcaf-770bc8938466 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:07.506682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5793cf-dcda-4f60-9288-4c8ac1e63cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:07.507307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5793cf-dcda-4f60-9288-4c8ac1e63cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.507590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5793cf-dcda-4f60-9288-4c8ac1e63cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.507867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5793cf-dcda-4f60-9288-4c8ac1e63cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.508543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5793cf-dcda-4f60-9288-4c8ac1e63cc9 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:07.508843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5793cf-dcda-4f60-9288-4c8ac1e63cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:07.510533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5793cf-dcda-4f60-9288-4c8ac1e63cc9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:07.519710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 142/288] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 19 02:12:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:07.525181 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfee7d4-29e6-41a8-ae4a-51dcb343487f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:07.536278 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfee7d4-29e6-41a8-ae4a-51dcb343487f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:07.540110 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5793cf-dcda-4f60-9288-4c8ac1e63cc9 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:07.551139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5793cf-dcda-4f60-9288-4c8ac1e63cc9 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:07.552943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d993d-9326-425a-a067-b0211ef9e3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:07.553835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d993d-9326-425a-a067-b0211ef9e3dc admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.554367 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d993d-9326-425a-a067-b0211ef9e3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.554843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d993d-9326-425a-a067-b0211ef9e3dc admin admin] PATH_INFO: `/v3/groups/63a99c8172a74137b1e60407401036bc` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.563562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d993d-9326-425a-a067-b0211ef9e3dc admin admin] RBAC: Authorizing `identity:get_group(group_id=63a99c8172a74137b1e60407401036bc)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:07.565302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: WARNING py.warnings [None req-488d993d-9326-425a-a067-b0211ef9e3dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:12:07.565302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: warnings.warn(msg) Oct 19 02:12:07.565302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: Oct 19 02:12:07.566271 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cda7e0-15b2-4281-800b-d445e65a92ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:07.567125 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cda7e0-15b2-4281-800b-d445e65a92ff admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.567594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cda7e0-15b2-4281-800b-d445e65a92ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.568055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cda7e0-15b2-4281-800b-d445e65a92ff admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.570236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d993d-9326-425a-a067-b0211ef9e3dc admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:07.573310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 147/289] 10.222.0.20 () {56 vars in 1201 bytes} [Wed Oct 19 02:12:07 2022] GET /identity/v3/groups/63a99c8172a74137b1e60407401036bc => generated 229 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:07.582995 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d993d-9326-425a-a067-b0211ef9e3dc admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:07.592794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:07.602094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d993d-9326-425a-a067-b0211ef9e3dc admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:07.602771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cda7e0-15b2-4281-800b-d445e65a92ff admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-b2cda7e0-15b2-4281-800b-d445e65a92ff admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:07.608823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:07.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:07.612543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 143/290] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 19 02:12:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 70 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:07.613892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e27fef-827a-483c-b1a5-8f73fad859d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:07.621621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e27fef-827a-483c-b1a5-8f73fad859d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.621621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e27fef-827a-483c-b1a5-8f73fad859d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.621621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e27fef-827a-483c-b1a5-8f73fad859d1 admin admin] PATH_INFO: `/v3/projects/984205d302cd4aa3b16c6c1e0788c92c` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.622270 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cda7e0-15b2-4281-800b-d445e65a92ff admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:07.622421 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e27fef-827a-483c-b1a5-8f73fad859d1 admin admin] RBAC: Authorizing `identity:get_project(project_id=984205d302cd4aa3b16c6c1e0788c92c)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:07.622421 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e27fef-827a-483c-b1a5-8f73fad859d1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:07.622421 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 148/291] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 19 02:12:07 2022] GET /identity/v3/projects/984205d302cd4aa3b16c6c1e0788c92c => generated 344 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:07.628250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e27fef-827a-483c-b1a5-8f73fad859d1 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:07.640638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cda7e0-15b2-4281-800b-d445e65a92ff admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:07.648462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129778b-7b9f-4472-bbe5-0a587d396eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:07.649420 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129778b-7b9f-4472-bbe5-0a587d396eaf admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.649916 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129778b-7b9f-4472-bbe5-0a587d396eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.650448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129778b-7b9f-4472-bbe5-0a587d396eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.651416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129778b-7b9f-4472-bbe5-0a587d396eaf admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:07.651918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129778b-7b9f-4472-bbe5-0a587d396eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:07.653825 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129778b-7b9f-4472-bbe5-0a587d396eaf admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:07.656307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e27fef-827a-483c-b1a5-8f73fad859d1 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:07.665876 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 144/292] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Oct 19 02:12:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:07.676066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129778b-7b9f-4472-bbe5-0a587d396eaf admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:07.677620 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd3ddf-9d11-4c9d-be34-047ce5ff5e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:07.678361 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd3ddf-9d11-4c9d-be34-047ce5ff5e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.678641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd3ddf-9d11-4c9d-be34-047ce5ff5e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.678903 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd3ddf-9d11-4c9d-be34-047ce5ff5e27 admin admin] PATH_INFO: `/v3/projects/984205d302cd4aa3b16c6c1e0788c92c/groups/63a99c8172a74137b1e60407401036bc/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.682317 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd3ddf-9d11-4c9d-be34-047ce5ff5e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=984205d302cd4aa3b16c6c1e0788c92c, group_id=63a99c8172a74137b1e60407401036bc, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:07.686748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd3ddf-9d11-4c9d-be34-047ce5ff5e27 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:07.693521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129778b-7b9f-4472-bbe5-0a587d396eaf admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:07.698953 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509fcc7-1ce9-4109-a188-968f0155c4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:07.699555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509fcc7-1ce9-4109-a188-968f0155c4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.699882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509fcc7-1ce9-4109-a188-968f0155c4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.700158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509fcc7-1ce9-4109-a188-968f0155c4b6 admin admin] PATH_INFO: `/v3/groups/22cc5bce0f1f40448cd8041a36289b21` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.701376 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509fcc7-1ce9-4109-a188-968f0155c4b6 admin admin] RBAC: Authorizing `identity:get_group(group_id=22cc5bce0f1f40448cd8041a36289b21)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:07.703492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509fcc7-1ce9-4109-a188-968f0155c4b6 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:07.705149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 145/293] 10.222.0.20 () {56 vars in 1201 bytes} [Wed Oct 19 02:12:07 2022] GET /identity/v3/groups/22cc5bce0f1f40448cd8041a36289b21 => generated 226 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:07.710533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509fcc7-1ce9-4109-a188-968f0155c4b6 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:07.721891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509fcc7-1ce9-4109-a188-968f0155c4b6 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:07.726643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58839035-8c81-427f-bd34-e496031c00c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:07.727273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58839035-8c81-427f-bd34-e496031c00c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.727595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58839035-8c81-427f-bd34-e496031c00c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.727876 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58839035-8c81-427f-bd34-e496031c00c3 admin admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.729111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58839035-8c81-427f-bd34-e496031c00c3 admin admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:07.731388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58839035-8c81-427f-bd34-e496031c00c3 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:07.733827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 146/294] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Oct 19 02:12:07 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:07.739310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58839035-8c81-427f-bd34-e496031c00c3 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:07.751389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58839035-8c81-427f-bd34-e496031c00c3 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:07.756163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb3418-5587-47d2-9808-36e8dc2d6da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:07.759778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb3418-5587-47d2-9808-36e8dc2d6da8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:07.763759 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 149/295] 10.222.0.20 () {58 vars in 1463 bytes} [Wed Oct 19 02:12:07 2022] PUT /identity/v3/projects/984205d302cd4aa3b16c6c1e0788c92c/groups/63a99c8172a74137b1e60407401036bc/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:07.764047 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb3418-5587-47d2-9808-36e8dc2d6da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:07.764321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb3418-5587-47d2-9808-36e8dc2d6da8 admin admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b/groups/22cc5bce0f1f40448cd8041a36289b21/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:07.766754 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb3418-5587-47d2-9808-36e8dc2d6da8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31fe422931242de9e41c5dc9136868b, group_id=22cc5bce0f1f40448cd8041a36289b21, role_id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:07.769683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb3418-5587-47d2-9808-36e8dc2d6da8 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:07.866359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 147/296] 10.222.0.20 () {58 vars in 1463 bytes} [Wed Oct 19 02:12:07 2022] PUT /identity/v3/projects/b31fe422931242de9e41c5dc9136868b/groups/22cc5bce0f1f40448cd8041a36289b21/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:12:08.340051 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7eb062-599d-4d6c-adda-770b0e84fba9 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:08.340437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7eb062-599d-4d6c-adda-770b0e84fba9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:08.340615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7eb062-599d-4d6c-adda-770b0e84fba9 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:08.342204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 150/297] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:08.347769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6586-949c-4b03-a7b5-7675f5fa62b9 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:08.348135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6586-949c-4b03-a7b5-7675f5fa62b9 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:08.348421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a6586-949c-4b03-a7b5-7675f5fa62b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:08.370420 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-904a6586-949c-4b03-a7b5-7675f5fa62b9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:08.371547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-904a6586-949c-4b03-a7b5-7675f5fa62b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:08.434183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200403b-3cb7-43fc-acb9-1ca204f4ccbf None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:08.434502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200403b-3cb7-43fc-acb9-1ca204f4ccbf None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:08.434731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200403b-3cb7-43fc-acb9-1ca204f4ccbf None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:08.436244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 151/298] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:08.441830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550a99-56dc-4098-a3ff-4d8cedea4475 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:08.442142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550a99-56dc-4098-a3ff-4d8cedea4475 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:08.442419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f550a99-56dc-4098-a3ff-4d8cedea4475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:08.471292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-7f550a99-56dc-4098-a3ff-4d8cedea4475 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:08.472495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7f550a99-56dc-4098-a3ff-4d8cedea4475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:08.492117 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 148/299] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:08.568514 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 152/300] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:12:08.909038 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a6586-949c-4b03-a7b5-7675f5fa62b9 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:08.940708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f550a99-56dc-4098-a3ff-4d8cedea4475 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:08.978418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:09.050743 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a6586-949c-4b03-a7b5-7675f5fa62b9 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:09.115595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7808f1b-c858-4fdd-b921-f3cf4a3067ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:09.117030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7808f1b-c858-4fdd-b921-f3cf4a3067ce None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:09.117309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7808f1b-c858-4fdd-b921-f3cf4a3067ce None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:09.117581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7808f1b-c858-4fdd-b921-f3cf4a3067ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:09.125093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f550a99-56dc-4098-a3ff-4d8cedea4475 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:09.147597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7808f1b-c858-4fdd-b921-f3cf4a3067ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-f7808f1b-c858-4fdd-b921-f3cf4a3067ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:09.152832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:09.153688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:09.154968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 149/301] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:09.164208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7808f1b-c858-4fdd-b921-f3cf4a3067ce None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:09.165653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:09.202628 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6ef28-dc75-4959-9580-0e201783d0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:09.203286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ef28-dc75-4959-9580-0e201783d0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:09.203567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ef28-dc75-4959-9580-0e201783d0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:09.203839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ef28-dc75-4959-9580-0e201783d0a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:09.223750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6ef28-dc75-4959-9580-0e201783d0a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-02c6ef28-dc75-4959-9580-0e201783d0a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:09.228656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:09.229846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:12:09.230653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:09.234092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 153/302] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:12:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:09.242066 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6ef28-dc75-4959-9580-0e201783d0a3 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:09.357365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7808f1b-c858-4fdd-b921-f3cf4a3067ce None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:09.378599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:09.402737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73334c1-1455-44fb-9303-b246878c538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:09.403622 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73334c1-1455-44fb-9303-b246878c538f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:09.406835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73334c1-1455-44fb-9303-b246878c538f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:09.407272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73334c1-1455-44fb-9303-b246878c538f None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:09.408098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73334c1-1455-44fb-9303-b246878c538f None admin] RBAC: Adding query filter params (name=member) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:09.408549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73334c1-1455-44fb-9303-b246878c538f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:09.411132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73334c1-1455-44fb-9303-b246878c538f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:09.430540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 150/303] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:12:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:09.436358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6ef28-dc75-4959-9580-0e201783d0a3 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:09.439478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73334c1-1455-44fb-9303-b246878c538f None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:09.502899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7d1ab-7ae1-4491-982f-245251edbf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:09.503502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7d1ab-7ae1-4491-982f-245251edbf02 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:09.503774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7d1ab-7ae1-4491-982f-245251edbf02 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:09.504038 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7d1ab-7ae1-4491-982f-245251edbf02 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:09.504696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7d1ab-7ae1-4491-982f-245251edbf02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:09.504994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7d1ab-7ae1-4491-982f-245251edbf02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:09.508725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7d1ab-7ae1-4491-982f-245251edbf02 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:09.525396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 154/304] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:09.531602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7d1ab-7ae1-4491-982f-245251edbf02 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:09.576780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73334c1-1455-44fb-9303-b246878c538f None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:09.594419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:09.648829 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46879373-d24f-4e5f-aa00-4bad00f8424b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:09.649490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46879373-d24f-4e5f-aa00-4bad00f8424b None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:09.649780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46879373-d24f-4e5f-aa00-4bad00f8424b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:09.650060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46879373-d24f-4e5f-aa00-4bad00f8424b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:09.686431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7d1ab-7ae1-4491-982f-245251edbf02 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:09.729583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:09.743622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23deb0c-aa57-4984-8096-d9dc2a7af95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:09.744236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23deb0c-aa57-4984-8096-d9dc2a7af95d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:09.744513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23deb0c-aa57-4984-8096-d9dc2a7af95d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:09.744779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23deb0c-aa57-4984-8096-d9dc2a7af95d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:09.753950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46879373-d24f-4e5f-aa00-4bad00f8424b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:09.755464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46879373-d24f-4e5f-aa00-4bad00f8424b None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:09.869288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23deb0c-aa57-4984-8096-d9dc2a7af95d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:09.870903 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23deb0c-aa57-4984-8096-d9dc2a7af95d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:09.915579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 151/305] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 19 02:12:09.916919 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cb8d1-3a9a-453f-ae3a-072d0d295c8d None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:09.917205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cb8d1-3a9a-453f-ae3a-072d0d295c8d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:09.917479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cb8d1-3a9a-453f-ae3a-072d0d295c8d None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:09.918784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 152/306] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:09.922898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cb8d1-3a9a-453f-ae3a-072d0d295c8d None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:09.927663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:09.989839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 155/307] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 19 02:12:09.991117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc82358-9da6-4211-b11d-61b6883bc5e8 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:09.991521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc82358-9da6-4211-b11d-61b6883bc5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:09.991688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc82358-9da6-4211-b11d-61b6883bc5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.011663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:10.019361 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-1dc82358-9da6-4211-b11d-61b6883bc5e8 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:10.020810 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1dc82358-9da6-4211-b11d-61b6883bc5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:10.034648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15cb8d1-3a9a-453f-ae3a-072d0d295c8d None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:10.069994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24460f4-09f3-46ed-9fee-fedd10cb445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:10.070668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24460f4-09f3-46ed-9fee-fedd10cb445d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.070945 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24460f4-09f3-46ed-9fee-fedd10cb445d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.071212 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24460f4-09f3-46ed-9fee-fedd10cb445d None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.072333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24460f4-09f3-46ed-9fee-fedd10cb445d None admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:10.073769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24460f4-09f3-46ed-9fee-fedd10cb445d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:10.075422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 153/308] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:12:09 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:10.078159 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24460f4-09f3-46ed-9fee-fedd10cb445d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:10.148045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 156/309] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:10.149110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc82358-9da6-4211-b11d-61b6883bc5e8 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:10.256532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24460f4-09f3-46ed-9fee-fedd10cb445d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:10.279067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:10.318970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96531f-2340-4192-9e3f-9cb8bab27400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:10.319617 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96531f-2340-4192-9e3f-9cb8bab27400 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.319896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96531f-2340-4192-9e3f-9cb8bab27400 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.320168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96531f-2340-4192-9e3f-9cb8bab27400 None admin] PATH_INFO: `/v3/projects/dce320a8ccf64f479c1dae1117865baf` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.321516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96531f-2340-4192-9e3f-9cb8bab27400 None admin] RBAC: Authorizing `identity:get_project(project_id=dce320a8ccf64f479c1dae1117865baf)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:10.324436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96531f-2340-4192-9e3f-9cb8bab27400 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:10.327007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 154/310] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:12:10 2022] GET /identity/v3/projects/dce320a8ccf64f479c1dae1117865baf => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:10.332522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96531f-2340-4192-9e3f-9cb8bab27400 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:10.334081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc82358-9da6-4211-b11d-61b6883bc5e8 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:10.360165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:10.397603 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784258e-f994-4ea6-a803-462252f8a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:10.400069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784258e-f994-4ea6-a803-462252f8a6bb None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.400353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784258e-f994-4ea6-a803-462252f8a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.400625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784258e-f994-4ea6-a803-462252f8a6bb None admin] PATH_INFO: `/v3/groups/22cc5bce0f1f40448cd8041a36289b21` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.405082 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784258e-f994-4ea6-a803-462252f8a6bb None admin] RBAC: Authorizing `identity:get_group(group_id=22cc5bce0f1f40448cd8041a36289b21)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:10.406701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784258e-f994-4ea6-a803-462252f8a6bb None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:10.410206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 157/311] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Oct 19 02:12:10 2022] GET /identity/v3/groups/22cc5bce0f1f40448cd8041a36289b21 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:10.419662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784258e-f994-4ea6-a803-462252f8a6bb None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:10.518099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96531f-2340-4192-9e3f-9cb8bab27400 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:10.569994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff4d6c-b15b-46ab-9845-412485035253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:10.570616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4d6c-b15b-46ab-9845-412485035253 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.570895 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4d6c-b15b-46ab-9845-412485035253 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.571168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff4d6c-b15b-46ab-9845-412485035253 None admin] PATH_INFO: `/v3/groups/22cc5bce0f1f40448cd8041a36289b21` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.572290 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff4d6c-b15b-46ab-9845-412485035253 None admin] RBAC: Authorizing `identity:get_group(group_id=22cc5bce0f1f40448cd8041a36289b21)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:10.573732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff4d6c-b15b-46ab-9845-412485035253 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:10.575649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 155/312] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Oct 19 02:12:10 2022] GET /identity/v3/groups/22cc5bce0f1f40448cd8041a36289b21 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:10.576957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff4d6c-b15b-46ab-9845-412485035253 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:10.589764 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784258e-f994-4ea6-a803-462252f8a6bb None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:10.626483 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c9207-37fe-47d7-947f-5e43bdce1054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:10.627331 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9207-37fe-47d7-947f-5e43bdce1054 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.627753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9207-37fe-47d7-947f-5e43bdce1054 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.628194 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c9207-37fe-47d7-947f-5e43bdce1054 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.629069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c9207-37fe-47d7-947f-5e43bdce1054 None admin] RBAC: Adding query filter params (group.id=22cc5bce0f1f40448cd8041a36289b21, role.id=bb36a823b2ff45d29018aedf19ae7bf0, scope.project.id=dce320a8ccf64f479c1dae1117865baf) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:10.629590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c9207-37fe-47d7-947f-5e43bdce1054 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:10.631381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c9207-37fe-47d7-947f-5e43bdce1054 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:10.637763 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 158/313] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Oct 19 02:12:10 2022] GET /identity/v3/role_assignments?group.id=22cc5bce0f1f40448cd8041a36289b21&scope.project.id=dce320a8ccf64f479c1dae1117865baf&role.id=bb36a823b2ff45d29018aedf19ae7bf0 => generated 611 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:10.638825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c9207-37fe-47d7-947f-5e43bdce1054 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:10.643782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:10.691662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff4d6c-b15b-46ab-9845-412485035253 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:10.693600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:10.754827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c921364-bfb4-4c8f-b726-53beb95a3a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:10.755656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c921364-bfb4-4c8f-b726-53beb95a3a67 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.756030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c921364-bfb4-4c8f-b726-53beb95a3a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.756397 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c921364-bfb4-4c8f-b726-53beb95a3a67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.767547 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c9207-37fe-47d7-947f-5e43bdce1054 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:10.772638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c921364-bfb4-4c8f-b726-53beb95a3a67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-0c921364-bfb4-4c8f-b726-53beb95a3a67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:10.779877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:10.781019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:10.782028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 156/314] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:10.782723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018af15f-be19-4ffb-9c9e-da20759f3fdc None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.782723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018af15f-be19-4ffb-9c9e-da20759f3fdc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.782723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018af15f-be19-4ffb-9c9e-da20759f3fdc None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.784018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 157/315] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:10.788617 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018af15f-be19-4ffb-9c9e-da20759f3fdc None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:10.810551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a11fe-ba93-45ba-bb31-dc46e7aa6ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:10.811547 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a11fe-ba93-45ba-bb31-dc46e7aa6ceb None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.812027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a11fe-ba93-45ba-bb31-dc46e7aa6ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.812473 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a11fe-ba93-45ba-bb31-dc46e7aa6ceb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.813412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a11fe-ba93-45ba-bb31-dc46e7aa6ceb None admin] RBAC: Adding query filter params (group.id=22cc5bce0f1f40448cd8041a36289b21, role.id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6, scope.project.id=dce320a8ccf64f479c1dae1117865baf) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:10.813922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a11fe-ba93-45ba-bb31-dc46e7aa6ceb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:10.816297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a11fe-ba93-45ba-bb31-dc46e7aa6ceb None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:10.823459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 159/316] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Oct 19 02:12:10 2022] GET /identity/v3/role_assignments?group.id=22cc5bce0f1f40448cd8041a36289b21&scope.project.id=dce320a8ccf64f479c1dae1117865baf&role.id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:10.825000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285795-68a7-400c-94ed-3c8c40520c7e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.825462 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285795-68a7-400c-94ed-3c8c40520c7e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.825909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26285795-68a7-400c-94ed-3c8c40520c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.848335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-26285795-68a7-400c-94ed-3c8c40520c7e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:10.849946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-26285795-68a7-400c-94ed-3c8c40520c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:10.920632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018af15f-be19-4ffb-9c9e-da20759f3fdc None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:10.921458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:10.931098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 160/317] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:10.931960 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a494c-1d4d-4a9a-982a-965c2afa8912 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.931960 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a494c-1d4d-4a9a-982a-965c2afa8912 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.932282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a494c-1d4d-4a9a-982a-965c2afa8912 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.933717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 161/318] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:10.940074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c49850-25e1-4d5a-bf1f-afb6da290f14 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.940319 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c49850-25e1-4d5a-bf1f-afb6da290f14 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.940562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c49850-25e1-4d5a-bf1f-afb6da290f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.959478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-98c49850-25e1-4d5a-bf1f-afb6da290f14 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:10.960708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-98c49850-25e1-4d5a-bf1f-afb6da290f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:10.981924 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8dce8-19b5-4da9-9287-c4d8d4382f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:10.982944 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8dce8-19b5-4da9-9287-c4d8d4382f77 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:10.983465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8dce8-19b5-4da9-9287-c4d8d4382f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:10.983961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8dce8-19b5-4da9-9287-c4d8d4382f77 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:10.984806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8dce8-19b5-4da9-9287-c4d8d4382f77 None admin] RBAC: Adding query filter params (name=member) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:10.985277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8dce8-19b5-4da9-9287-c4d8d4382f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:10.987010 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8dce8-19b5-4da9-9287-c4d8d4382f77 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:11.005480 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 158/319] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:12:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:11.011549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8dce8-19b5-4da9-9287-c4d8d4382f77 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:11.020574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:11.037670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 162/320] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:12:11.152638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8dce8-19b5-4da9-9287-c4d8d4382f77 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:11.189120 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fc7c1-2857-40ee-b862-283c70766086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:11.189725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fc7c1-2857-40ee-b862-283c70766086 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:11.189996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fc7c1-2857-40ee-b862-283c70766086 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:11.190319 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fc7c1-2857-40ee-b862-283c70766086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:11.258972 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c49850-25e1-4d5a-bf1f-afb6da290f14 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:11.268957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fc7c1-2857-40ee-b862-283c70766086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:11.270910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fc7c1-2857-40ee-b862-283c70766086 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:11.273977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:11.372087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c49850-25e1-4d5a-bf1f-afb6da290f14 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:11.392624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 159/321] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 19 02:12:11.393474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fc7c1-2857-40ee-b862-283c70766086 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:11.406607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:11.419243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853c155-7a98-4354-89ec-4169eef93b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:11.425708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853c155-7a98-4354-89ec-4169eef93b09 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:11.425708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853c155-7a98-4354-89ec-4169eef93b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:11.425708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853c155-7a98-4354-89ec-4169eef93b09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:11.431609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853c155-7a98-4354-89ec-4169eef93b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-d853c155-7a98-4354-89ec-4169eef93b09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:11.436828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:11.437776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:11.439376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 163/322] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:12:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:11.440203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853c155-7a98-4354-89ec-4169eef93b09 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:11.511424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fc7c1-2857-40ee-b862-283c70766086 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:11.515609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:11.549975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041bb1f-dab4-4fc6-89c1-1675b86d8dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:11.550645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041bb1f-dab4-4fc6-89c1-1675b86d8dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:11.550967 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041bb1f-dab4-4fc6-89c1-1675b86d8dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:11.551279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853c155-7a98-4354-89ec-4169eef93b09 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:11.551412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041bb1f-dab4-4fc6-89c1-1675b86d8dd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:11.563400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041bb1f-dab4-4fc6-89c1-1675b86d8dd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-5041bb1f-dab4-4fc6-89c1-1675b86d8dd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:11.568399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:11.569289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:11.570087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 160/323] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:12:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:11.571204 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041bb1f-dab4-4fc6-89c1-1675b86d8dd2 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:11.587741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a914446-29a5-45d0-bcc1-00358fe0fc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:11.588363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a914446-29a5-45d0-bcc1-00358fe0fc30 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:11.588640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a914446-29a5-45d0-bcc1-00358fe0fc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:11.588914 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a914446-29a5-45d0-bcc1-00358fe0fc30 None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:11.590034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a914446-29a5-45d0-bcc1-00358fe0fc30 None admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:11.591509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a914446-29a5-45d0-bcc1-00358fe0fc30 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:11.593344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 164/324] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:12:11 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:11.594980 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a914446-29a5-45d0-bcc1-00358fe0fc30 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:11.656233 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:11.702844 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041bb1f-dab4-4fc6-89c1-1675b86d8dd2 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:11.709349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a914446-29a5-45d0-bcc1-00358fe0fc30 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:11.746022 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e765653-bbaa-4cee-96de-53cf919d35d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:11.746954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e765653-bbaa-4cee-96de-53cf919d35d4 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:11.747069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce9c89-6352-4d06-b24c-a91ebe1c84e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:11.747495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e765653-bbaa-4cee-96de-53cf919d35d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:11.747624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce9c89-6352-4d06-b24c-a91ebe1c84e1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:11.747796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e765653-bbaa-4cee-96de-53cf919d35d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:11.747902 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce9c89-6352-4d06-b24c-a91ebe1c84e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:11.748090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce9c89-6352-4d06-b24c-a91ebe1c84e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:11.748974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e765653-bbaa-4cee-96de-53cf919d35d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:11.749280 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e765653-bbaa-4cee-96de-53cf919d35d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:11.749378 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce9c89-6352-4d06-b24c-a91ebe1c84e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:11.749581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce9c89-6352-4d06-b24c-a91ebe1c84e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:11.750693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e765653-bbaa-4cee-96de-53cf919d35d4 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:11.751063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce9c89-6352-4d06-b24c-a91ebe1c84e1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:11.759534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 161/325] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:11.760359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e765653-bbaa-4cee-96de-53cf919d35d4 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:11.760627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 165/326] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:11.766269 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce9c89-6352-4d06-b24c-a91ebe1c84e1 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:11.791770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:11.878567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e765653-bbaa-4cee-96de-53cf919d35d4 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:11.883124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce9c89-6352-4d06-b24c-a91ebe1c84e1 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:11.899541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:11.921065 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5cf2d-4978-4cdd-86b5-d7f806441499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:11.921627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5cf2d-4978-4cdd-86b5-d7f806441499 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:11.921886 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5cf2d-4978-4cdd-86b5-d7f806441499 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:11.922187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5cf2d-4978-4cdd-86b5-d7f806441499 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:11.925001 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a16954-b1cc-4d15-9eb7-07d7a1802dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:11.925647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a16954-b1cc-4d15-9eb7-07d7a1802dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:11.925929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a16954-b1cc-4d15-9eb7-07d7a1802dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:11.926241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a16954-b1cc-4d15-9eb7-07d7a1802dc0 None admin] PATH_INFO: `/v3/groups/22cc5bce0f1f40448cd8041a36289b21` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:11.927381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a16954-b1cc-4d15-9eb7-07d7a1802dc0 None admin] RBAC: Authorizing `identity:get_group(group_id=22cc5bce0f1f40448cd8041a36289b21)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:11.928819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a16954-b1cc-4d15-9eb7-07d7a1802dc0 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:11.930444 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 162/327] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Oct 19 02:12:11 2022] GET /identity/v3/groups/22cc5bce0f1f40448cd8041a36289b21 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:11.931221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a16954-b1cc-4d15-9eb7-07d7a1802dc0 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:12.003295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5cf2d-4978-4cdd-86b5-d7f806441499 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:12.004969 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5cf2d-4978-4cdd-86b5-d7f806441499 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:12.035903 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:12.042491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a16954-b1cc-4d15-9eb7-07d7a1802dc0 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:12.075753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912edd0-36ce-4adf-80e5-d93eca668544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:12.076873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912edd0-36ce-4adf-80e5-d93eca668544 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:12.077168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912edd0-36ce-4adf-80e5-d93eca668544 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:12.077441 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912edd0-36ce-4adf-80e5-d93eca668544 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:12.114452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 166/328] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 19 02:12:12.118976 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5cf2d-4978-4cdd-86b5-d7f806441499 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:12.135342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:12.145379 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912edd0-36ce-4adf-80e5-d93eca668544 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:12.146852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912edd0-36ce-4adf-80e5-d93eca668544 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:12.216961 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5cf2d-4978-4cdd-86b5-d7f806441499 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:12.238473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:12.245405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 163/329] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:12:12.246246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912edd0-36ce-4adf-80e5-d93eca668544 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:12.253713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01affd58-8e9b-4f69-bbfa-c1615cdda395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:12.254338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01affd58-8e9b-4f69-bbfa-c1615cdda395 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:12.254613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01affd58-8e9b-4f69-bbfa-c1615cdda395 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:12.254881 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01affd58-8e9b-4f69-bbfa-c1615cdda395 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:12.255599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01affd58-8e9b-4f69-bbfa-c1615cdda395 None admin] RBAC: Adding query filter params (group.id=22cc5bce0f1f40448cd8041a36289b21, role.id=bb36a823b2ff45d29018aedf19ae7bf0, scope.project.id=b31fe422931242de9e41c5dc9136868b) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:12.255899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01affd58-8e9b-4f69-bbfa-c1615cdda395 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:12.257319 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01affd58-8e9b-4f69-bbfa-c1615cdda395 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:12.263087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 167/330] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Oct 19 02:12:12 2022] GET /identity/v3/role_assignments?group.id=22cc5bce0f1f40448cd8041a36289b21&scope.project.id=b31fe422931242de9e41c5dc9136868b&role.id=bb36a823b2ff45d29018aedf19ae7bf0 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:12.263841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01affd58-8e9b-4f69-bbfa-c1615cdda395 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:12.351790 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:12.367159 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912edd0-36ce-4adf-80e5-d93eca668544 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:12.372809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01affd58-8e9b-4f69-bbfa-c1615cdda395 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:12.408184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339d171-dc0f-4457-96fb-e76a9ded2b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:12.408935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339d171-dc0f-4457-96fb-e76a9ded2b2c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:12.409211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339d171-dc0f-4457-96fb-e76a9ded2b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:12.409473 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46289b27-b4b9-4788-8811-ec398d917798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:12.409604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e339d171-dc0f-4457-96fb-e76a9ded2b2c None admin] PATH_INFO: `/v3/projects/984205d302cd4aa3b16c6c1e0788c92c` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:12.410145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46289b27-b4b9-4788-8811-ec398d917798 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:12.410433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46289b27-b4b9-4788-8811-ec398d917798 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:12.410676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339d171-dc0f-4457-96fb-e76a9ded2b2c None admin] RBAC: Authorizing `identity:get_project(project_id=984205d302cd4aa3b16c6c1e0788c92c)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:12.410770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46289b27-b4b9-4788-8811-ec398d917798 None admin] PATH_INFO: `/v3/projects/b31fe422931242de9e41c5dc9136868b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:12.411824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46289b27-b4b9-4788-8811-ec398d917798 None admin] RBAC: Authorizing `identity:get_project(project_id=b31fe422931242de9e41c5dc9136868b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:12.412137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e339d171-dc0f-4457-96fb-e76a9ded2b2c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:12.413249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46289b27-b4b9-4788-8811-ec398d917798 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:12.413991 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 164/331] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:12:12 2022] GET /identity/v3/projects/984205d302cd4aa3b16c6c1e0788c92c => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:12.414916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 168/332] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:12:12 2022] GET /identity/v3/projects/b31fe422931242de9e41c5dc9136868b => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:12.419468 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339d171-dc0f-4457-96fb-e76a9ded2b2c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:12.423191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46289b27-b4b9-4788-8811-ec398d917798 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:12.486649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:12.534756 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e339d171-dc0f-4457-96fb-e76a9ded2b2c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:12.546486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46289b27-b4b9-4788-8811-ec398d917798 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:12.576222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9392f50-cf6b-4816-b776-01c979d7d3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:12.576907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9392f50-cf6b-4816-b776-01c979d7d3ad None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:12.577179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9392f50-cf6b-4816-b776-01c979d7d3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:12.577445 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9392f50-cf6b-4816-b776-01c979d7d3ad None admin] PATH_INFO: `/v3/groups/63a99c8172a74137b1e60407401036bc` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:12.578565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9392f50-cf6b-4816-b776-01c979d7d3ad None admin] RBAC: Authorizing `identity:get_group(group_id=63a99c8172a74137b1e60407401036bc)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:12.579972 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9392f50-cf6b-4816-b776-01c979d7d3ad None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:12.581689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 165/333] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Oct 19 02:12:12 2022] GET /identity/v3/groups/63a99c8172a74137b1e60407401036bc => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:12.588015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9392f50-cf6b-4816-b776-01c979d7d3ad None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:12.589475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d892e0-26fd-40b1-a930-ddf11e4de8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:12.590076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d892e0-26fd-40b1-a930-ddf11e4de8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:12.590346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d892e0-26fd-40b1-a930-ddf11e4de8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:12.590910 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d892e0-26fd-40b1-a930-ddf11e4de8f8 None admin] PATH_INFO: `/v3/groups/22cc5bce0f1f40448cd8041a36289b21` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:12.591975 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d892e0-26fd-40b1-a930-ddf11e4de8f8 None admin] RBAC: Authorizing `identity:get_group(group_id=22cc5bce0f1f40448cd8041a36289b21)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:12.593800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d892e0-26fd-40b1-a930-ddf11e4de8f8 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:12.601381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 169/334] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Oct 19 02:12:12 2022] GET /identity/v3/groups/22cc5bce0f1f40448cd8041a36289b21 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:12.601740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d892e0-26fd-40b1-a930-ddf11e4de8f8 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:12.620969 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:12.698401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9392f50-cf6b-4816-b776-01c979d7d3ad None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:12.708584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d892e0-26fd-40b1-a930-ddf11e4de8f8 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:12.726267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:12.732056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005ec5b-c49d-4fe1-96e0-9eec492c5b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:12.738258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005ec5b-c49d-4fe1-96e0-9eec492c5b8d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:12.738258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005ec5b-c49d-4fe1-96e0-9eec492c5b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:12.738258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005ec5b-c49d-4fe1-96e0-9eec492c5b8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:12.738258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005ec5b-c49d-4fe1-96e0-9eec492c5b8d None admin] RBAC: Adding query filter params (group.id=63a99c8172a74137b1e60407401036bc, role.id=d5070bb151d343ca8de32abe373f4a73, scope.project.id=984205d302cd4aa3b16c6c1e0788c92c) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:12.738258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005ec5b-c49d-4fe1-96e0-9eec492c5b8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:12.738258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005ec5b-c49d-4fe1-96e0-9eec492c5b8d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:12.741611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f316c3-2015-42b1-94b6-6cd638f6a7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:12.742115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 166/335] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Oct 19 02:12:12 2022] GET /identity/v3/role_assignments?group.id=63a99c8172a74137b1e60407401036bc&scope.project.id=984205d302cd4aa3b16c6c1e0788c92c&role.id=d5070bb151d343ca8de32abe373f4a73 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:12.742220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f316c3-2015-42b1-94b6-6cd638f6a7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:12.742501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f316c3-2015-42b1-94b6-6cd638f6a7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:12.742774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f316c3-2015-42b1-94b6-6cd638f6a7c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:12.743494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f316c3-2015-42b1-94b6-6cd638f6a7c5 None admin] RBAC: Adding query filter params (group.id=22cc5bce0f1f40448cd8041a36289b21, role.id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6, scope.project.id=b31fe422931242de9e41c5dc9136868b) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:12.743818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f316c3-2015-42b1-94b6-6cd638f6a7c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:12.745347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f316c3-2015-42b1-94b6-6cd638f6a7c5 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:12.752307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 170/336] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Oct 19 02:12:12 2022] GET /identity/v3/role_assignments?group.id=22cc5bce0f1f40448cd8041a36289b21&scope.project.id=b31fe422931242de9e41c5dc9136868b&role.id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:15.426144 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b584-5d79-4b74-8498-525fdb4d73a6 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:15.426678 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b584-5d79-4b74-8498-525fdb4d73a6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:15.426678 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b584-5d79-4b74-8498-525fdb4d73a6 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:15.428225 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 167/337] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:15.433391 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8440155-c9d1-425c-a91a-01f640830e0b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:15.433836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8440155-c9d1-425c-a91a-01f640830e0b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:15.433918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8440155-c9d1-425c-a91a-01f640830e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:15.458083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-d8440155-c9d1-425c-a91a-01f640830e0b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:15.459261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d8440155-c9d1-425c-a91a-01f640830e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:15.546727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 171/338] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:15.759523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0a020-aef8-4afc-baf6-c5e30edf29d2 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:15.759913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0a020-aef8-4afc-baf6-c5e30edf29d2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:15.760094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0a020-aef8-4afc-baf6-c5e30edf29d2 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:15.761645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 168/339] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:15.767201 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33975e38-c3e8-4bc8-91dc-137962fb950c None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:15.767484 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33975e38-c3e8-4bc8-91dc-137962fb950c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:15.767759 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33975e38-c3e8-4bc8-91dc-137962fb950c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:15.791088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-33975e38-c3e8-4bc8-91dc-137962fb950c None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:15.796341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-33975e38-c3e8-4bc8-91dc-137962fb950c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:15.796341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:15.875853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 172/340] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:12:15.891565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0a020-aef8-4afc-baf6-c5e30edf29d2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:16.001673 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a7b29-52ef-4336-b024-4b01dca9ac57 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:16.002019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a7b29-52ef-4336-b024-4b01dca9ac57 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:16.002262 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a7b29-52ef-4336-b024-4b01dca9ac57 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:16.003932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 173/341] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:16.009919 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8761a-1297-4b78-b045-0a76f8fc3d9e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:16.010238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8761a-1297-4b78-b045-0a76f8fc3d9e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:16.010424 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8761a-1297-4b78-b045-0a76f8fc3d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:16.036022 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0a020-aef8-4afc-baf6-c5e30edf29d2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:16.044124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-5fb8761a-1297-4b78-b045-0a76f8fc3d9e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:16.045941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5fb8761a-1297-4b78-b045-0a76f8fc3d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:16.105194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358a2f8-337a-4060-92df-131132c428f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:16.105847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358a2f8-337a-4060-92df-131132c428f6 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:16.106153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358a2f8-337a-4060-92df-131132c428f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:16.106428 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358a2f8-337a-4060-92df-131132c428f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:16.133836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358a2f8-337a-4060-92df-131132c428f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:16.135321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358a2f8-337a-4060-92df-131132c428f6 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:16.156922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 174/342] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-4358a2f8-337a-4060-92df-131132c428f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:16.158089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:16.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:16.159900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:16.159900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:16.159900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:16.159900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 169/343] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:16.161003 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ee9a4-f10e-48ee-8087-5615b6a1777b None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:16.161499 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ee9a4-f10e-48ee-8087-5615b6a1777b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:16.162007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ee9a4-f10e-48ee-8087-5615b6a1777b None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:16.163750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 175/344] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:16.167826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358a2f8-337a-4060-92df-131132c428f6 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:16.169171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c1b7a-ec15-4f78-a283-35abd89ff4f2 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:16.169710 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c1b7a-ec15-4f78-a283-35abd89ff4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:16.170208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c1b7a-ec15-4f78-a283-35abd89ff4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:16.203007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-111c1b7a-ec15-4f78-a283-35abd89ff4f2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:16.204497 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-111c1b7a-ec15-4f78-a283-35abd89ff4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:16.224983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:16.286345 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358a2f8-337a-4060-92df-131132c428f6 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:16.287924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:16.293120 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 176/345] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:16.294178 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c1b7a-ec15-4f78-a283-35abd89ff4f2 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:16.322655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1e708-0598-48ea-9fca-66875139de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:16.323262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1e708-0598-48ea-9fca-66875139de4e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:16.323539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1e708-0598-48ea-9fca-66875139de4e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:16.323815 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1e708-0598-48ea-9fca-66875139de4e None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:16.324458 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1e708-0598-48ea-9fca-66875139de4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:16.324768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1e708-0598-48ea-9fca-66875139de4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:16.326547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1e708-0598-48ea-9fca-66875139de4e None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:16.342667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 170/346] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:16.345395 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1e708-0598-48ea-9fca-66875139de4e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:16.445093 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1e708-0598-48ea-9fca-66875139de4e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:16.456544 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:16.489305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcff8a-d27a-4442-ba07-f6c73dd668a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:16.489919 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcff8a-d27a-4442-ba07-f6c73dd668a9 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:16.490220 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcff8a-d27a-4442-ba07-f6c73dd668a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:16.490492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcff8a-d27a-4442-ba07-f6c73dd668a9 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:16.491147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcff8a-d27a-4442-ba07-f6c73dd668a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:16.492666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcff8a-d27a-4442-ba07-f6c73dd668a9 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:16.516434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c1b7a-ec15-4f78-a283-35abd89ff4f2 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:16.553293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93f078-727f-4d05-bb38-3eefb43c5fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:16.554172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93f078-727f-4d05-bb38-3eefb43c5fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:16.554672 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93f078-727f-4d05-bb38-3eefb43c5fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:16.555118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93f078-727f-4d05-bb38-3eefb43c5fc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:16.566789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93f078-727f-4d05-bb38-3eefb43c5fc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:16.568879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93f078-727f-4d05-bb38-3eefb43c5fc6 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:16.577293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 171/347] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:12:16 2022] POST /identity/v3/users => generated 246 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:12:16.578645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcff8a-d27a-4442-ba07-f6c73dd668a9 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-7a93f078-727f-4d05-bb38-3eefb43c5fc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:16.609232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:16.610231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:16.611112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:16.611112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:16.611112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:16.612899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 177/348] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:16.619069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93f078-727f-4d05-bb38-3eefb43c5fc6 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:16.693704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:16.730538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcff8a-d27a-4442-ba07-f6c73dd668a9 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:16.767106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93f078-727f-4d05-bb38-3eefb43c5fc6 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:16.768897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c08f2-b410-4146-bb8a-fcd7fa11bef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:16.769541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c08f2-b410-4146-bb8a-fcd7fa11bef7 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:16.769821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c08f2-b410-4146-bb8a-fcd7fa11bef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:16.770156 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c08f2-b410-4146-bb8a-fcd7fa11bef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:16.782146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c08f2-b410-4146-bb8a-fcd7fa11bef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:16.783553 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c08f2-b410-4146-bb8a-fcd7fa11bef7 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-ec9c08f2-b410-4146-bb8a-fcd7fa11bef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:16.802629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:16.803812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:16.804828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:16.804828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:16.804828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:16.804828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 172/349] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:16.805054 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c08f2-b410-4146-bb8a-fcd7fa11bef7 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:16.813762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68ceb0-ac90-4d46-9586-ad7af2b89666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:16.814472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68ceb0-ac90-4d46-9586-ad7af2b89666 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:16.814750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68ceb0-ac90-4d46-9586-ad7af2b89666 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:16.815022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68ceb0-ac90-4d46-9586-ad7af2b89666 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:16.815669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68ceb0-ac90-4d46-9586-ad7af2b89666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:16.816431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68ceb0-ac90-4d46-9586-ad7af2b89666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:16.817817 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68ceb0-ac90-4d46-9586-ad7af2b89666 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:16.825487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:16.832107 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 178/350] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:16.835574 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68ceb0-ac90-4d46-9586-ad7af2b89666 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:16.921894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c08f2-b410-4146-bb8a-fcd7fa11bef7 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:16.958731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e6020-3e94-4321-86c1-fc87ae5c9a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:16.959358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e6020-3e94-4321-86c1-fc87ae5c9a57 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:16.959639 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e6020-3e94-4321-86c1-fc87ae5c9a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:16.959911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e6020-3e94-4321-86c1-fc87ae5c9a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:16.966170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68ceb0-ac90-4d46-9586-ad7af2b89666 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:16.973875 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e6020-3e94-4321-86c1-fc87ae5c9a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:16.975380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e6020-3e94-4321-86c1-fc87ae5c9a57 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:16.987372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-d58e6020-3e94-4321-86c1-fc87ae5c9a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:16.991670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:16.992532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:16.993340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:16.993340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:16.993340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:16.993528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 173/351] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:16.994371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e6020-3e94-4321-86c1-fc87ae5c9a57 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:17.017948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687996d-4363-478f-858c-51aaf0cd7e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:17.018938 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687996d-4363-478f-858c-51aaf0cd7e5e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:17.019472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687996d-4363-478f-858c-51aaf0cd7e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:17.019984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687996d-4363-478f-858c-51aaf0cd7e5e None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:17.021378 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687996d-4363-478f-858c-51aaf0cd7e5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:17.022211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687996d-4363-478f-858c-51aaf0cd7e5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:17.024150 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687996d-4363-478f-858c-51aaf0cd7e5e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:17.045424 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 179/352] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:17.046780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687996d-4363-478f-858c-51aaf0cd7e5e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:17.071655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:17.112512 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e6020-3e94-4321-86c1-fc87ae5c9a57 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:17.150725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc77243-1e2f-4a34-be27-5f38293a6520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:17.151266 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc77243-1e2f-4a34-be27-5f38293a6520 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:17.151600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc77243-1e2f-4a34-be27-5f38293a6520 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:17.151960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc77243-1e2f-4a34-be27-5f38293a6520 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:17.152984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc77243-1e2f-4a34-be27-5f38293a6520 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:17.153997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687996d-4363-478f-858c-51aaf0cd7e5e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:17.154383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc77243-1e2f-4a34-be27-5f38293a6520 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:17.188926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100526e-3103-42b4-8a93-657ad120c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:17.189605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100526e-3103-42b4-8a93-657ad120c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:17.189873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100526e-3103-42b4-8a93-657ad120c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:17.190159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100526e-3103-42b4-8a93-657ad120c7c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:17.190791 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100526e-3103-42b4-8a93-657ad120c7c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:17.191087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100526e-3103-42b4-8a93-657ad120c7c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:17.192451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2100526e-3103-42b4-8a93-657ad120c7c6 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:17.209307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 180/353] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:17.209307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100526e-3103-42b4-8a93-657ad120c7c6 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:17.238722 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:17.262356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 174/354] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:12:16 2022] POST /identity/v3/users => generated 248 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:17.263367 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc77243-1e2f-4a34-be27-5f38293a6520 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:17.334778 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2100526e-3103-42b4-8a93-657ad120c7c6 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:17.381126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea2aff-3d0a-4e24-94cc-c84ee2f78814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:17.382157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea2aff-3d0a-4e24-94cc-c84ee2f78814 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:17.383316 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea2aff-3d0a-4e24-94cc-c84ee2f78814 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:17.383839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea2aff-3d0a-4e24-94cc-c84ee2f78814 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:17.384709 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea2aff-3d0a-4e24-94cc-c84ee2f78814 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:17.386524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea2aff-3d0a-4e24-94cc-c84ee2f78814 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:17.471724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc77243-1e2f-4a34-be27-5f38293a6520 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:17.497318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:17.510444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 181/355] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:12:17 2022] POST /identity/v3/users => generated 248 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:17.511935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e8eed-a623-4297-bc79-04854967e4aa None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:17.512281 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e8eed-a623-4297-bc79-04854967e4aa None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:17.512619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e8eed-a623-4297-bc79-04854967e4aa None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:17.514467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 182/356] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:17.518664 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279035e-676a-4f04-b8ec-6ba1f8f93711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:17.519295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279035e-676a-4f04-b8ec-6ba1f8f93711 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:17.519575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279035e-676a-4f04-b8ec-6ba1f8f93711 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:17.520384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279035e-676a-4f04-b8ec-6ba1f8f93711 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:17.521093 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279035e-676a-4f04-b8ec-6ba1f8f93711 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:17.521597 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4b5e4-670d-4888-bbb0-0f0b1cfa8c34 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:17.522018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4b5e4-670d-4888-bbb0-0f0b1cfa8c34 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:17.522275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4b5e4-670d-4888-bbb0-0f0b1cfa8c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:17.522605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279035e-676a-4f04-b8ec-6ba1f8f93711 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:17.546375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c4f4b5e4-670d-4888-bbb0-0f0b1cfa8c34 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:17.547610 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c4f4b5e4-670d-4888-bbb0-0f0b1cfa8c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:17.621926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 175/357] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:12:17 2022] POST /identity/v3/users => generated 249 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:17.650577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 183/358] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:12:17.974595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279035e-676a-4f04-b8ec-6ba1f8f93711 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:18.085390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279035e-676a-4f04-b8ec-6ba1f8f93711 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:18.128941 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443c606-35ba-4659-b38a-e955e787724c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:18.129813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443c606-35ba-4659-b38a-e955e787724c None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:18.130292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443c606-35ba-4659-b38a-e955e787724c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:18.130727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443c606-35ba-4659-b38a-e955e787724c None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:18.131573 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443c606-35ba-4659-b38a-e955e787724c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:18.133076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443c606-35ba-4659-b38a-e955e787724c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:18.207501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.sql.core [None req-f443c606-35ba-4659-b38a-e955e787724c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 19 02:12:18.207501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 19 02:12:18.207501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [parameters: {'id': '323e151ca7eb464988c1ce6b0fa34f67', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 19 02:12:18.207501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81616) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-f443c606-35ba-4659-b38a-e955e787724c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application conn.query(q) Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result.read() Oct 19 02:12:18.215509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return ref.to_dict() Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 02:12:18.216583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application next(self.gen) Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application yield resource Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application next(self.gen) Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.session.rollback() Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.force_reraise() Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise self.value Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application session.commit() Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 19 02:12:18.217770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._prepare_impl() Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.session.flush() Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._flush(objects) Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application compat.raise_( Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application flush_context.execute() Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rec.execute(self) Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 19 02:12:18.223014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 02:12:18.224308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application conn.query(q) Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result.read() Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 02:12:18.230358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [parameters: {'id': '323e151ca7eb464988c1ce6b0fa34f67', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:18.231470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 19 02:12:18.232555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:18.234296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 176/359] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Oct 19 02:12:17 2022] POST /identity/v3/roles => generated 144 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:18.237763 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4b5e4-670d-4888-bbb0-0f0b1cfa8c34 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:18.257085 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:18.343720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4b5e4-670d-4888-bbb0-0f0b1cfa8c34 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:18.385951 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4556f1-c3db-4fd7-bcb0-9c92788280b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:18.386887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4556f1-c3db-4fd7-bcb0-9c92788280b3 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:18.387348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4556f1-c3db-4fd7-bcb0-9c92788280b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:18.387770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4556f1-c3db-4fd7-bcb0-9c92788280b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:18.403584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4556f1-c3db-4fd7-bcb0-9c92788280b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-ea4556f1-c3db-4fd7-bcb0-9c92788280b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:18.409446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:18.410577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 19 02:12:18.411585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:18.413878 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 184/360] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:12:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:18.419978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443c606-35ba-4659-b38a-e955e787724c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:18.476484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:18.556658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443c606-35ba-4659-b38a-e955e787724c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:18.594570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaecfb7-a4f7-4529-a205-c859fe96e20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:18.595187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaecfb7-a4f7-4529-a205-c859fe96e20c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:18.595468 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaecfb7-a4f7-4529-a205-c859fe96e20c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:18.595741 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaecfb7-a4f7-4529-a205-c859fe96e20c None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:18.596404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaecfb7-a4f7-4529-a205-c859fe96e20c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:18.596705 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaecfb7-a4f7-4529-a205-c859fe96e20c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:18.598085 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaecfb7-a4f7-4529-a205-c859fe96e20c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:18.606443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 177/361] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:18.978284 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f95e-4756-4f6b-871c-62a7e7aeb4bc None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:18.979130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f95e-4756-4f6b-871c-62a7e7aeb4bc None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:18.979571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f95e-4756-4f6b-871c-62a7e7aeb4bc None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:18.981665 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 185/362] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:18.987912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b810ee-26cd-4d8c-8fdb-d45566a6692a None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:18.988374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b810ee-26cd-4d8c-8fdb-d45566a6692a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:18.988813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b810ee-26cd-4d8c-8fdb-d45566a6692a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:19.010453 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-86b810ee-26cd-4d8c-8fdb-d45566a6692a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:19.011830 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-86b810ee-26cd-4d8c-8fdb-d45566a6692a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:19.078394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 178/363] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:19.362801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595f95e-4756-4f6b-871c-62a7e7aeb4bc None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:19.447820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:19.471999 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595f95e-4756-4f6b-871c-62a7e7aeb4bc None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:19.515286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54e656-e180-4b3a-92f0-17ba98255724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:19.515966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54e656-e180-4b3a-92f0-17ba98255724 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:19.516249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54e656-e180-4b3a-92f0-17ba98255724 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:19.516524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54e656-e180-4b3a-92f0-17ba98255724 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:19.527119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54e656-e180-4b3a-92f0-17ba98255724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-fb54e656-e180-4b3a-92f0-17ba98255724 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:19.532590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:19.533457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:19.534576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 186/364] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:19.543396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b810ee-26cd-4d8c-8fdb-d45566a6692a None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:19.585796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:19.670783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b810ee-26cd-4d8c-8fdb-d45566a6692a None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:19.723667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547c126-308a-470d-833c-bba7630341af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:19.724301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547c126-308a-470d-833c-bba7630341af None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:19.724585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547c126-308a-470d-833c-bba7630341af None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:19.724860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547c126-308a-470d-833c-bba7630341af None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:19.725545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547c126-308a-470d-833c-bba7630341af None admin] RBAC: Adding query filter params (name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:19.725851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547c126-308a-470d-833c-bba7630341af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:19.727301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547c126-308a-470d-833c-bba7630341af None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:19.739219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 179/365] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:19.744886 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54e656-e180-4b3a-92f0-17ba98255724 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:19.855323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad526d8-48b4-4d0a-ac3d-56e966c4aab7 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:19.855663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad526d8-48b4-4d0a-ac3d-56e966c4aab7 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:19.855875 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad526d8-48b4-4d0a-ac3d-56e966c4aab7 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:19.857448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 180/366] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:19.862298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a7899-e577-4b19-a6c1-60107db11abe None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:19.862578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a7899-e577-4b19-a6c1-60107db11abe None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:19.862846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a7899-e577-4b19-a6c1-60107db11abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:19.875907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54e656-e180-4b3a-92f0-17ba98255724 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:19.898082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ed2a7899-e577-4b19-a6c1-60107db11abe None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:19.899275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ed2a7899-e577-4b19-a6c1-60107db11abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:19.913176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd330957-9256-4505-85ad-cf55041b5b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:19.913935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd330957-9256-4505-85ad-cf55041b5b26 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:19.914308 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd330957-9256-4505-85ad-cf55041b5b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:19.914647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd330957-9256-4505-85ad-cf55041b5b26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:19.932088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:19.939768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd330957-9256-4505-85ad-cf55041b5b26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:19.941248 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd330957-9256-4505-85ad-cf55041b5b26 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-dd330957-9256-4505-85ad-cf55041b5b26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:19.962209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:19.963437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:19.964955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:19.964955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:19.964955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:19.964955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 187/367] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:19.969870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd330957-9256-4505-85ad-cf55041b5b26 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:20.079708 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 181/368] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:20.084808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd330957-9256-4505-85ad-cf55041b5b26 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:20.132152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca77a93-d32d-44c8-a4d6-07fb7412318a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:20.132823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca77a93-d32d-44c8-a4d6-07fb7412318a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:20.133098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca77a93-d32d-44c8-a4d6-07fb7412318a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:20.133369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca77a93-d32d-44c8-a4d6-07fb7412318a None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:20.134089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca77a93-d32d-44c8-a4d6-07fb7412318a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:20.134402 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca77a93-d32d-44c8-a4d6-07fb7412318a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:20.135886 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca77a93-d32d-44c8-a4d6-07fb7412318a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:20.148669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 188/369] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:20.153992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a7899-e577-4b19-a6c1-60107db11abe None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:20.186476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:20.256456 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a7899-e577-4b19-a6c1-60107db11abe None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:20.295899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc3d32-9a9a-4cd0-b95c-d75d753ad02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:20.296565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc3d32-9a9a-4cd0-b95c-d75d753ad02f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:20.296912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc3d32-9a9a-4cd0-b95c-d75d753ad02f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:20.297194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc3d32-9a9a-4cd0-b95c-d75d753ad02f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:20.310463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc3d32-9a9a-4cd0-b95c-d75d753ad02f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:20.312060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc3d32-9a9a-4cd0-b95c-d75d753ad02f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-74fc3d32-9a9a-4cd0-b95c-d75d753ad02f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:20.332835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:20.333785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:20.334804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:20.334804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:20.334804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:20.334804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:20.334804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:20.334804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:20.334804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:20.334804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:20.334804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:12:20.334804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:20.334804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 182/370] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 19 02:12:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:20.341906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca77a93-d32d-44c8-a4d6-07fb7412318a None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:20.395699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:20.428709 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eabc7-1231-4dba-8979-203c5afc1605 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:20.429007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eabc7-1231-4dba-8979-203c5afc1605 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:20.429249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eabc7-1231-4dba-8979-203c5afc1605 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:20.430599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 183/371] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:20.435720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18fae3-2a09-4096-b668-a248256aa006 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:20.436058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18fae3-2a09-4096-b668-a248256aa006 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:20.436325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18fae3-2a09-4096-b668-a248256aa006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:20.455116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca77a93-d32d-44c8-a4d6-07fb7412318a None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:20.474606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-8f18fae3-2a09-4096-b668-a248256aa006 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:20.475788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8f18fae3-2a09-4096-b668-a248256aa006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:20.493958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874977d-d45a-482d-bd9d-936cf6846ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:20.494657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874977d-d45a-482d-bd9d-936cf6846ceb None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:20.495292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874977d-d45a-482d-bd9d-936cf6846ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:20.495669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874977d-d45a-482d-bd9d-936cf6846ceb None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:20.496418 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874977d-d45a-482d-bd9d-936cf6846ceb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:20.502066 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874977d-d45a-482d-bd9d-936cf6846ceb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:20.502066 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874977d-d45a-482d-bd9d-936cf6846ceb None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:20.548347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 189/372] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 19 02:12:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:20.553449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874977d-d45a-482d-bd9d-936cf6846ceb None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:20.570186 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 184/373] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:20.571265 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18fae3-2a09-4096-b668-a248256aa006 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:20.618409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:20.673303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874977d-d45a-482d-bd9d-936cf6846ceb None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:20.684192 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:20.711540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c73d1-8592-4392-b6e8-03d1e72ba455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:20.713668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c73d1-8592-4392-b6e8-03d1e72ba455 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:20.714190 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c73d1-8592-4392-b6e8-03d1e72ba455 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:20.714708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c73d1-8592-4392-b6e8-03d1e72ba455 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:20.726080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c73d1-8592-4392-b6e8-03d1e72ba455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-c76c73d1-8592-4392-b6e8-03d1e72ba455 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:20.731394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:20.732235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:20.733061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:20.735865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 190/374] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:20.737357 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf951f-40a4-4fc9-8c5f-9501d15b2164 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:20.737925 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf951f-40a4-4fc9-8c5f-9501d15b2164 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:20.738439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf951f-40a4-4fc9-8c5f-9501d15b2164 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:20.744116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 191/375] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:20.744243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18fae3-2a09-4096-b668-a248256aa006 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:20.745870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf951f-40a4-4fc9-8c5f-9501d15b2164 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:20.786236 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deeecb3-d958-41c8-9d94-9d2949c33b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:20.786872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deeecb3-d958-41c8-9d94-9d2949c33b69 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:20.787151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deeecb3-d958-41c8-9d94-9d2949c33b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:20.787422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deeecb3-d958-41c8-9d94-9d2949c33b69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:20.799616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deeecb3-d958-41c8-9d94-9d2949c33b69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:20.801373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deeecb3-d958-41c8-9d94-9d2949c33b69 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-8deeecb3-d958-41c8-9d94-9d2949c33b69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:20.820550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:20.821424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:20.822278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:20.822278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:20.822278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:20.822278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 185/376] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:20.823141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0294c-0635-4d3d-ae42-1f65e5ee2d59 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:20.823141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0294c-0635-4d3d-ae42-1f65e5ee2d59 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:20.823430 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b0294c-0635-4d3d-ae42-1f65e5ee2d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:20.846233 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-30b0294c-0635-4d3d-ae42-1f65e5ee2d59 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:20.847720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-30b0294c-0635-4d3d-ae42-1f65e5ee2d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:20.869845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf951f-40a4-4fc9-8c5f-9501d15b2164 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:20.907275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74f6a8-28f3-4431-a8c1-01ab9794f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:20.907897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74f6a8-28f3-4431-a8c1-01ab9794f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:20.908171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74f6a8-28f3-4431-a8c1-01ab9794f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:20.908426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74f6a8-28f3-4431-a8c1-01ab9794f7e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:20.909074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74f6a8-28f3-4431-a8c1-01ab9794f7e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:20.909369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74f6a8-28f3-4431-a8c1-01ab9794f7e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:20.910761 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74f6a8-28f3-4431-a8c1-01ab9794f7e0 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:20.922102 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 192/377] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:20.922102 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74f6a8-28f3-4431-a8c1-01ab9794f7e0 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:20.923203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:20.927969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 186/378] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:20.927969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0294c-0635-4d3d-ae42-1f65e5ee2d59 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:20.973175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:21.031641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b0294c-0635-4d3d-ae42-1f65e5ee2d59 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:21.066708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74f6a8-28f3-4431-a8c1-01ab9794f7e0 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:21.079421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42165e7-fe5e-4205-831b-d3b0ec7b952e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:21.080632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42165e7-fe5e-4205-831b-d3b0ec7b952e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.081157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42165e7-fe5e-4205-831b-d3b0ec7b952e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.081933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42165e7-fe5e-4205-831b-d3b0ec7b952e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.097846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42165e7-fe5e-4205-831b-d3b0ec7b952e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-d42165e7-fe5e-4205-831b-d3b0ec7b952e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:21.105077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:21.105936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:21.106784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:21.115084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 187/379] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:21.116416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42165e7-fe5e-4205-831b-d3b0ec7b952e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:21.120510 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0f8f0-637a-4dbe-8975-a2a50927a1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:21.121045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0f8f0-637a-4dbe-8975-a2a50927a1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.121252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0f8f0-637a-4dbe-8975-a2a50927a1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.121517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0f8f0-637a-4dbe-8975-a2a50927a1f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.122195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0f8f0-637a-4dbe-8975-a2a50927a1f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:21.122497 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0f8f0-637a-4dbe-8975-a2a50927a1f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:21.123944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0f8f0-637a-4dbe-8975-a2a50927a1f0 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:21.141055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 193/380] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:21.141779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0f8f0-637a-4dbe-8975-a2a50927a1f0 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:21.187322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:21.234337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42165e7-fe5e-4205-831b-d3b0ec7b952e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:21.235672 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:21.245931 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0f8f0-637a-4dbe-8975-a2a50927a1f0 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:21.270437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2f4c5-bb72-42c6-84fd-a6d558bfef8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:21.271063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2f4c5-bb72-42c6-84fd-a6d558bfef8d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.271337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2f4c5-bb72-42c6-84fd-a6d558bfef8d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.271810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2f4c5-bb72-42c6-84fd-a6d558bfef8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.281807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2f4c5-bb72-42c6-84fd-a6d558bfef8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:21.282173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c17b6-c439-4e3e-b551-d2bdd66a39ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:21.282780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c17b6-c439-4e3e-b551-d2bdd66a39ce None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.283144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c17b6-c439-4e3e-b551-d2bdd66a39ce None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.283304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2f4c5-bb72-42c6-84fd-a6d558bfef8d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:21.283403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c17b6-c439-4e3e-b551-d2bdd66a39ce None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.284060 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c17b6-c439-4e3e-b551-d2bdd66a39ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:21.284363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c17b6-c439-4e3e-b551-d2bdd66a39ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:21.285714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c17b6-c439-4e3e-b551-d2bdd66a39ce None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:21.298089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 194/381] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:21.298089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c17b6-c439-4e3e-b551-d2bdd66a39ce None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-c3f2f4c5-bb72-42c6-84fd-a6d558bfef8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:21.299334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:21.300219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:21.301037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:21.301037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:21.301037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:21.301037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 188/382] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:21.301037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2f4c5-bb72-42c6-84fd-a6d558bfef8d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:21.409079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c17b6-c439-4e3e-b551-d2bdd66a39ce None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:21.413010 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2f4c5-bb72-42c6-84fd-a6d558bfef8d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:21.424419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:21.444020 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c975a-b7f0-4ed0-ae0a-8863b03187d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:21.444646 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c975a-b7f0-4ed0-ae0a-8863b03187d9 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.444924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c975a-b7f0-4ed0-ae0a-8863b03187d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.445199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c975a-b7f0-4ed0-ae0a-8863b03187d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.449304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e201a-cf92-4b8b-bd02-fcfeeb2c55d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:21.450067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e201a-cf92-4b8b-bd02-fcfeeb2c55d3 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.450450 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e201a-cf92-4b8b-bd02-fcfeeb2c55d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.450819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e201a-cf92-4b8b-bd02-fcfeeb2c55d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.452734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c975a-b7f0-4ed0-ae0a-8863b03187d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:21.455013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c975a-b7f0-4ed0-ae0a-8863b03187d9 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:21.463127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e201a-cf92-4b8b-bd02-fcfeeb2c55d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-cd6c975a-b7f0-4ed0-ae0a-8863b03187d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:21.467890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:21.468891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:21.469913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:21.469913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:21.469913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:21.469913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:21.469913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:21.469913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:21.469913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:21.469913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-679e201a-cf92-4b8b-bd02-fcfeeb2c55d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:21.470470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:21.471486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:21.472474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:21.473227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 195/383] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:21.473227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c975a-b7f0-4ed0-ae0a-8863b03187d9 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:21.479148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 189/384] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:21.479148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e201a-cf92-4b8b-bd02-fcfeeb2c55d3 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:21.488175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:21.570856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c975a-b7f0-4ed0-ae0a-8863b03187d9 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:21.571948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e201a-cf92-4b8b-bd02-fcfeeb2c55d3 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:21.604314 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696949fb-6390-43ec-9c98-6af5da8982b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:21.604947 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696949fb-6390-43ec-9c98-6af5da8982b0 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.605222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696949fb-6390-43ec-9c98-6af5da8982b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.605487 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696949fb-6390-43ec-9c98-6af5da8982b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.605601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ceb1f-11e4-4b85-9020-bf26854d4890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:21.606188 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ceb1f-11e4-4b85-9020-bf26854d4890 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.606469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ceb1f-11e4-4b85-9020-bf26854d4890 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.606743 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ceb1f-11e4-4b85-9020-bf26854d4890 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.607381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ceb1f-11e4-4b85-9020-bf26854d4890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:21.607678 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ceb1f-11e4-4b85-9020-bf26854d4890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:21.609072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ceb1f-11e4-4b85-9020-bf26854d4890 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:21.614987 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696949fb-6390-43ec-9c98-6af5da8982b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:21.616407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696949fb-6390-43ec-9c98-6af5da8982b0 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:21.620135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 190/385] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:21.625190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e14cd-a97e-429c-b60d-d0b3c29cc05f None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.625190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e14cd-a97e-429c-b60d-d0b3c29cc05f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.625190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e14cd-a97e-429c-b60d-d0b3c29cc05f None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.625190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 191/386] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:21.625190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e14cd-a97e-429c-b60d-d0b3c29cc05f None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-696949fb-6390-43ec-9c98-6af5da8982b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:21.633891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:21.635027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:21.635897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:21.635897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:21.635897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:21.635897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 196/387] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:21.635897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696949fb-6390-43ec-9c98-6af5da8982b0 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:21.660273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:21.696500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:21.731081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e14cd-a97e-429c-b60d-d0b3c29cc05f None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:21.732617 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696949fb-6390-43ec-9c98-6af5da8982b0 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:21.765147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f897d65-5675-49da-9fb8-241403dfaae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:21.765736 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f897d65-5675-49da-9fb8-241403dfaae1 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.766009 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f897d65-5675-49da-9fb8-241403dfaae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.766301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f897d65-5675-49da-9fb8-241403dfaae1 None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.766657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3462c-27c6-4051-ab71-5fb1922cc45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:21.766982 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f897d65-5675-49da-9fb8-241403dfaae1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:21.767257 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3462c-27c6-4051-ab71-5fb1922cc45b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.767355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f897d65-5675-49da-9fb8-241403dfaae1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:21.767536 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3462c-27c6-4051-ab71-5fb1922cc45b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.767997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3462c-27c6-4051-ab71-5fb1922cc45b None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.768666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f897d65-5675-49da-9fb8-241403dfaae1 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:21.769731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3462c-27c6-4051-ab71-5fb1922cc45b None admin] RBAC: Adding query filter params (name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:21.770071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3462c-27c6-4051-ab71-5fb1922cc45b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:21.771437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3462c-27c6-4051-ab71-5fb1922cc45b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:21.784737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 197/388] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:21.785804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3462c-27c6-4051-ab71-5fb1922cc45b None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:21.800499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 192/389] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:21.800499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadba6a-181a-4b4e-af04-e4b61bf0e8fd None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.800499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadba6a-181a-4b4e-af04-e4b61bf0e8fd None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.800499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aadba6a-181a-4b4e-af04-e4b61bf0e8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.817245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9aadba6a-181a-4b4e-af04-e4b61bf0e8fd None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:21.818466 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9aadba6a-181a-4b4e-af04-e4b61bf0e8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:21.884104 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 193/390] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:21.884926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadba6a-181a-4b4e-af04-e4b61bf0e8fd None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:21.887875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3462c-27c6-4051-ab71-5fb1922cc45b None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:21.914956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:21.915968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:21.925088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208c481-a43e-4a1c-b55a-63a323555be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:21.927555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208c481-a43e-4a1c-b55a-63a323555be1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:21.927555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208c481-a43e-4a1c-b55a-63a323555be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:21.927555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208c481-a43e-4a1c-b55a-63a323555be1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:21.934911 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208c481-a43e-4a1c-b55a-63a323555be1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:21.937430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208c481-a43e-4a1c-b55a-63a323555be1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-0208c481-a43e-4a1c-b55a-63a323555be1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:21.952295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:21.953203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:21.954147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:21.954147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:21.954147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:21.954147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:21.954147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:21.954147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:21.954147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:21.954147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:21.954147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 02:12:21.954147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:21.954147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 198/391] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:21.954688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208c481-a43e-4a1c-b55a-63a323555be1 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:22.005324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aadba6a-181a-4b4e-af04-e4b61bf0e8fd None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:22.041170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4a774-c63a-4882-a127-e88f445612c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:22.041678 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4a774-c63a-4882-a127-e88f445612c7 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:22.041952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4a774-c63a-4882-a127-e88f445612c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:22.042258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4a774-c63a-4882-a127-e88f445612c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:22.042892 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4a774-c63a-4882-a127-e88f445612c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:22.043185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4a774-c63a-4882-a127-e88f445612c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:22.044543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4a774-c63a-4882-a127-e88f445612c7 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:22.061564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 194/392] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:22.061564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4a774-c63a-4882-a127-e88f445612c7 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:22.061815 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208c481-a43e-4a1c-b55a-63a323555be1 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:22.098103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e85dc5-96ee-4bfc-b06d-c4480ed1f3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:22.098303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e85dc5-96ee-4bfc-b06d-c4480ed1f3ee None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:22.098521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e85dc5-96ee-4bfc-b06d-c4480ed1f3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:22.102895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e85dc5-96ee-4bfc-b06d-c4480ed1f3ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:22.114328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e85dc5-96ee-4bfc-b06d-c4480ed1f3ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:22.115832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e85dc5-96ee-4bfc-b06d-c4480ed1f3ee None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-08e85dc5-96ee-4bfc-b06d-c4480ed1f3ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:22.133030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:22.134496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:22.135363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:22.135363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:22.135363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:22.135363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 199/393] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:22.139825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e85dc5-96ee-4bfc-b06d-c4480ed1f3ee None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:22.157049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:22.177653 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4a774-c63a-4882-a127-e88f445612c7 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:22.184990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:22.215862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8340285-d5af-4091-b074-09fe8088a485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:22.216523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8340285-d5af-4091-b074-09fe8088a485 None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:22.216796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8340285-d5af-4091-b074-09fe8088a485 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:22.217067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8340285-d5af-4091-b074-09fe8088a485 None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/0abf52399a344157ac4e0f9a8212164e/roles/695bad5e383d4052a3d6ffa680329cb7` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:22.232486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8340285-d5af-4091-b074-09fe8088a485 None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=0abf52399a344157ac4e0f9a8212164e, role_id=695bad5e383d4052a3d6ffa680329cb7)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:22.234136 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8340285-d5af-4091-b074-09fe8088a485 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:22.249275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e85dc5-96ee-4bfc-b06d-c4480ed1f3ee None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:22.284422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af11bd-4f18-42cb-afa6-94b76d8154fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:22.285035 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af11bd-4f18-42cb-afa6-94b76d8154fd None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:22.285306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af11bd-4f18-42cb-afa6-94b76d8154fd None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:22.285570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af11bd-4f18-42cb-afa6-94b76d8154fd None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:22.286315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af11bd-4f18-42cb-afa6-94b76d8154fd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:22.286614 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af11bd-4f18-42cb-afa6-94b76d8154fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:22.288070 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af11bd-4f18-42cb-afa6-94b76d8154fd None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:22.307297 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 195/394] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:12:22 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/0abf52399a344157ac4e0f9a8212164e/roles/695bad5e383d4052a3d6ffa680329cb7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:22.312817 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 200/395] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Oct 19 02:12:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:22.312979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8340285-d5af-4091-b074-09fe8088a485 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:22.313064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af11bd-4f18-42cb-afa6-94b76d8154fd None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:22.439064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:22.440574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8340285-d5af-4091-b074-09fe8088a485 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:22.444969 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af11bd-4f18-42cb-afa6-94b76d8154fd None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:22.474518 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:22.482241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925d5b-1f47-417b-8f9f-6503f8b1ffe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:22.483092 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925d5b-1f47-417b-8f9f-6503f8b1ffe1 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:22.483372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925d5b-1f47-417b-8f9f-6503f8b1ffe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:22.483650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af925d5b-1f47-417b-8f9f-6503f8b1ffe1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:22.491650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af925d5b-1f47-417b-8f9f-6503f8b1ffe1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:22.493099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af925d5b-1f47-417b-8f9f-6503f8b1ffe1 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:22.493227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45c39a-f9df-4b8f-8c5d-17cfb43860cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:22.493791 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45c39a-f9df-4b8f-8c5d-17cfb43860cf None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:22.494801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45c39a-f9df-4b8f-8c5d-17cfb43860cf None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:22.494801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45c39a-f9df-4b8f-8c5d-17cfb43860cf None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:22.494976 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45c39a-f9df-4b8f-8c5d-17cfb43860cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:22.495282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45c39a-f9df-4b8f-8c5d-17cfb43860cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:22.496659 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45c39a-f9df-4b8f-8c5d-17cfb43860cf None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:22.509435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 201/396] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:22.510360 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45c39a-f9df-4b8f-8c5d-17cfb43860cf None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-af925d5b-1f47-417b-8f9f-6503f8b1ffe1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:22.519960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:22.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:22.521796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:22.521796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:22.521796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:22.521796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:22.521796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:22.521796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:22.521796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:22.521796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:22.521796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 19 02:12:22.521796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:22.521796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 196/397] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:22.522882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925d5b-1f47-417b-8f9f-6503f8b1ffe1 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:22.630198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45c39a-f9df-4b8f-8c5d-17cfb43860cf None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:22.638884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af925d5b-1f47-417b-8f9f-6503f8b1ffe1 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:22.671558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37c37a-3690-4f22-87f9-85d6513e9ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:22.672348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37c37a-3690-4f22-87f9-85d6513e9ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:22.672537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37c37a-3690-4f22-87f9-85d6513e9ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:22.672883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37c37a-3690-4f22-87f9-85d6513e9ff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:22.675729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664b90f-7825-466d-b3ae-00782476c92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:22.676390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664b90f-7825-466d-b3ae-00782476c92e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:22.676845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664b90f-7825-466d-b3ae-00782476c92e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:22.677014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3664b90f-7825-466d-b3ae-00782476c92e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:22.684441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37c37a-3690-4f22-87f9-85d6513e9ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:22.685897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37c37a-3690-4f22-87f9-85d6513e9ff3 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:22.689124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664b90f-7825-466d-b3ae-00782476c92e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:22.690599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3664b90f-7825-466d-b3ae-00782476c92e None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-fb37c37a-3690-4f22-87f9-85d6513e9ff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:22.706728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:22.707644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:22.714034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:22.714034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:22.714034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-3664b90f-7825-466d-b3ae-00782476c92e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:22.714273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:22.715075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:22.715906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:22.715906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:22.715906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:22.716074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 202/398] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:22.716074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37c37a-3690-4f22-87f9-85d6513e9ff3 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:22.716190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 197/399] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:22.716190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664b90f-7825-466d-b3ae-00782476c92e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:22.728519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:22.769060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:22.822720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37c37a-3690-4f22-87f9-85d6513e9ff3 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:22.833897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3664b90f-7825-466d-b3ae-00782476c92e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:22.867978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02390ae6-c31b-4862-adc6-ba6dcfb96b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:22.868648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390ae6-c31b-4862-adc6-ba6dcfb96b92 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:22.868921 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390ae6-c31b-4862-adc6-ba6dcfb96b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:22.869254 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02390ae6-c31b-4862-adc6-ba6dcfb96b92 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:22.877288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02390ae6-c31b-4862-adc6-ba6dcfb96b92 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:22.878359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02390ae6-c31b-4862-adc6-ba6dcfb96b92 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:22.879160 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680c571-727b-4ec6-a1c4-d2c1ed4e981b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:22.880121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680c571-727b-4ec6-a1c4-d2c1ed4e981b None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:22.880686 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680c571-727b-4ec6-a1c4-d2c1ed4e981b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:22.881151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680c571-727b-4ec6-a1c4-d2c1ed4e981b None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:22.882036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680c571-727b-4ec6-a1c4-d2c1ed4e981b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:22.882508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680c571-727b-4ec6-a1c4-d2c1ed4e981b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:22.884086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680c571-727b-4ec6-a1c4-d2c1ed4e981b None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-02390ae6-c31b-4862-adc6-ba6dcfb96b92 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:22.890503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:22.891382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:22.892272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:22.892272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:22.892272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:22.892272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:22.892272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:22.892272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:22.892272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:22.892272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:22.892272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 19 02:12:22.892272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:22.892272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 203/400] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:22.892824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02390ae6-c31b-4862-adc6-ba6dcfb96b92 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:22.930103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 198/401] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Oct 19 02:12:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:22.930103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680c571-727b-4ec6-a1c4-d2c1ed4e981b None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:22.973109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:23.011205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02390ae6-c31b-4862-adc6-ba6dcfb96b92 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:23.032103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:23.035249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680c571-727b-4ec6-a1c4-d2c1ed4e981b None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:23.046183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696d6b1-108b-4ebe-872f-edd39050d65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:23.047022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696d6b1-108b-4ebe-872f-edd39050d65d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:23.047581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696d6b1-108b-4ebe-872f-edd39050d65d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:23.047996 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696d6b1-108b-4ebe-872f-edd39050d65d None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:23.048772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696d6b1-108b-4ebe-872f-edd39050d65d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:23.049224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696d6b1-108b-4ebe-872f-edd39050d65d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:23.050749 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696d6b1-108b-4ebe-872f-edd39050d65d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:23.064029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 204/402] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:23.064690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696d6b1-108b-4ebe-872f-edd39050d65d None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:23.074798 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233674c-6c5c-43ab-9c8e-ef43d1843045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:23.075637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233674c-6c5c-43ab-9c8e-ef43d1843045 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:23.076056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233674c-6c5c-43ab-9c8e-ef43d1843045 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:23.076495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233674c-6c5c-43ab-9c8e-ef43d1843045 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:23.077349 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233674c-6c5c-43ab-9c8e-ef43d1843045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:23.077652 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233674c-6c5c-43ab-9c8e-ef43d1843045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:23.079048 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d233674c-6c5c-43ab-9c8e-ef43d1843045 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:23.095631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 199/403] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:23.095631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233674c-6c5c-43ab-9c8e-ef43d1843045 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:23.178952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696d6b1-108b-4ebe-872f-edd39050d65d None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:23.198185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d233674c-6c5c-43ab-9c8e-ef43d1843045 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:23.206623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:23.215144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3058b1-fca7-4ad1-a82d-dc663c3394c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:23.215753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3058b1-fca7-4ad1-a82d-dc663c3394c8 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:23.216034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3058b1-fca7-4ad1-a82d-dc663c3394c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:23.216501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3058b1-fca7-4ad1-a82d-dc663c3394c8 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:23.224217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3058b1-fca7-4ad1-a82d-dc663c3394c8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:23.224217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3058b1-fca7-4ad1-a82d-dc663c3394c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:23.224217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3058b1-fca7-4ad1-a82d-dc663c3394c8 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:23.233162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4f464-2b5d-42f7-9e0c-9899a0dab799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:23.233827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4f464-2b5d-42f7-9e0c-9899a0dab799 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:23.234135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4f464-2b5d-42f7-9e0c-9899a0dab799 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:23.234408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4f464-2b5d-42f7-9e0c-9899a0dab799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:23.237671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 205/404] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Oct 19 02:12:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:23.238833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3058b1-fca7-4ad1-a82d-dc663c3394c8 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:23.252105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4f464-2b5d-42f7-9e0c-9899a0dab799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:23.253378 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4f464-2b5d-42f7-9e0c-9899a0dab799 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-5fb4f464-2b5d-42f7-9e0c-9899a0dab799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:23.268121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:23.269062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:23.274697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:23.274697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:23.274697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:23.274697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 200/405] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:23.274697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4f464-2b5d-42f7-9e0c-9899a0dab799 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:23.285594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:23.356492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3058b1-fca7-4ad1-a82d-dc663c3394c8 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:23.369265 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4f464-2b5d-42f7-9e0c-9899a0dab799 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:23.391351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35c4ad-ed3a-40f8-9679-08f9579ff73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:23.391968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35c4ad-ed3a-40f8-9679-08f9579ff73c None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:23.392250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35c4ad-ed3a-40f8-9679-08f9579ff73c None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:23.392517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35c4ad-ed3a-40f8-9679-08f9579ff73c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:23.399198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35c4ad-ed3a-40f8-9679-08f9579ff73c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:23.400636 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35c4ad-ed3a-40f8-9679-08f9579ff73c None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:23.403371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11838ae0-2fb2-4522-9adb-181f7c028542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:23.403964 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11838ae0-2fb2-4522-9adb-181f7c028542 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:23.404238 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11838ae0-2fb2-4522-9adb-181f7c028542 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:23.404506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11838ae0-2fb2-4522-9adb-181f7c028542 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:23.405198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11838ae0-2fb2-4522-9adb-181f7c028542 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:23.406648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11838ae0-2fb2-4522-9adb-181f7c028542 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-0d35c4ad-ed3a-40f8-9679-08f9579ff73c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:23.411858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:23.412679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:23.413501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:23.413501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:23.413501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:23.413501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:23.413501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:23.413501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:23.413501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:23.413501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:23.413501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 206/406] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:23.414525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35c4ad-ed3a-40f8-9679-08f9579ff73c None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:23.459316 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:23.510392 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35c4ad-ed3a-40f8-9679-08f9579ff73c None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:23.542082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 201/407] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:12:23 2022] POST /identity/v3/users => generated 247 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:23.542082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11838ae0-2fb2-4522-9adb-181f7c028542 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:23.552455 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a8eba-3ba5-43fc-8ba1-da74e848d6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:23.553316 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a8eba-3ba5-43fc-8ba1-da74e848d6ec None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:23.553561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a8eba-3ba5-43fc-8ba1-da74e848d6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:23.553906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a8eba-3ba5-43fc-8ba1-da74e848d6ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:23.571076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a8eba-3ba5-43fc-8ba1-da74e848d6ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:23.573501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a8eba-3ba5-43fc-8ba1-da74e848d6ec None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-834a8eba-3ba5-43fc-8ba1-da74e848d6ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:23.593625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:23.594800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:23.595794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:23.595794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:23.595794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:23.596225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 207/408] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:23.597385 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a8eba-3ba5-43fc-8ba1-da74e848d6ec None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:23.628378 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:23.675961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11838ae0-2fb2-4522-9adb-181f7c028542 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:23.720959 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:23.723329 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55e09b-3ff1-4053-9cc4-ec7667f2365f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:23.724352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55e09b-3ff1-4053-9cc4-ec7667f2365f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:23.724734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55e09b-3ff1-4053-9cc4-ec7667f2365f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:23.725116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e55e09b-3ff1-4053-9cc4-ec7667f2365f None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:23.725781 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55e09b-3ff1-4053-9cc4-ec7667f2365f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:23.726118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55e09b-3ff1-4053-9cc4-ec7667f2365f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:23.726216 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a8eba-3ba5-43fc-8ba1-da74e848d6ec None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:23.727655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e55e09b-3ff1-4053-9cc4-ec7667f2365f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:23.742019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 202/409] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:23.743163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55e09b-3ff1-4053-9cc4-ec7667f2365f None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:23.770218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad1946-62b5-46d2-ab20-a1aeaf9a014b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:23.770827 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad1946-62b5-46d2-ab20-a1aeaf9a014b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:23.771106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad1946-62b5-46d2-ab20-a1aeaf9a014b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:23.771406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad1946-62b5-46d2-ab20-a1aeaf9a014b None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:23.772173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad1946-62b5-46d2-ab20-a1aeaf9a014b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:23.772490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad1946-62b5-46d2-ab20-a1aeaf9a014b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:23.774648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad1946-62b5-46d2-ab20-a1aeaf9a014b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:23.793765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 208/410] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:23.794726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad1946-62b5-46d2-ab20-a1aeaf9a014b None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:23.860324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e55e09b-3ff1-4053-9cc4-ec7667f2365f None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:23.890816 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:23.900860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a3a7c-7f1e-4487-b654-6558d391e7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:23.901702 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3a7c-7f1e-4487-b654-6558d391e7fc None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:23.905254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3a7c-7f1e-4487-b654-6558d391e7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:23.905254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a3a7c-7f1e-4487-b654-6558d391e7fc None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:23.905254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a3a7c-7f1e-4487-b654-6558d391e7fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:23.905254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a3a7c-7f1e-4487-b654-6558d391e7fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:23.905254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a3a7c-7f1e-4487-b654-6558d391e7fc None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:23.907830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad1946-62b5-46d2-ab20-a1aeaf9a014b None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:23.916884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 203/411] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:23.917911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a3a7c-7f1e-4487-b654-6558d391e7fc None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:23.946083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0bbbee-b079-44d2-8053-0ac61a8f70f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:23.946823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bbbee-b079-44d2-8053-0ac61a8f70f0 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:23.947097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bbbee-b079-44d2-8053-0ac61a8f70f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:23.947364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0bbbee-b079-44d2-8053-0ac61a8f70f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:23.956776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0bbbee-b079-44d2-8053-0ac61a8f70f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:23.958276 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0bbbee-b079-44d2-8053-0ac61a8f70f0 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-1d0bbbee-b079-44d2-8053-0ac61a8f70f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:23.973328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:23.974213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:23.975055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:23.975055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:23.975055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:23.975055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:23.975055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:23.975055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:23.975055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:23.975055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:23.975055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 209/412] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:23.976053 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0bbbee-b079-44d2-8053-0ac61a8f70f0 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:24.000809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:24.033507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a3a7c-7f1e-4487-b654-6558d391e7fc None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:24.074502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71989eb-fb12-4e88-8dde-cbd67448130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:24.075320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71989eb-fb12-4e88-8dde-cbd67448130b None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:24.075596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71989eb-fb12-4e88-8dde-cbd67448130b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:24.076159 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71989eb-fb12-4e88-8dde-cbd67448130b None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/e63c5bd8c4264db697947f025c67f512/roles/695bad5e383d4052a3d6ffa680329cb7` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:24.093087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0bbbee-b079-44d2-8053-0ac61a8f70f0 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:24.095335 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71989eb-fb12-4e88-8dde-cbd67448130b None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=e63c5bd8c4264db697947f025c67f512, role_id=695bad5e383d4052a3d6ffa680329cb7)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:24.097567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71989eb-fb12-4e88-8dde-cbd67448130b None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:24.131441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5617e-4ca6-4af1-98f1-457d0eb06880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:24.132067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5617e-4ca6-4af1-98f1-457d0eb06880 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:24.132346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5617e-4ca6-4af1-98f1-457d0eb06880 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:24.132619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5617e-4ca6-4af1-98f1-457d0eb06880 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:24.139950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5617e-4ca6-4af1-98f1-457d0eb06880 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:24.141396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5617e-4ca6-4af1-98f1-457d0eb06880 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-edb5617e-4ca6-4af1-98f1-457d0eb06880 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:24.152734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:24.157458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:24.158321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:24.158321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:24.158321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:24.158321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:24.158321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:24.158321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:24.158321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:24.158321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:24.158321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 210/413] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:24.158321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5617e-4ca6-4af1-98f1-457d0eb06880 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:24.190949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 204/414] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:12:23 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/e63c5bd8c4264db697947f025c67f512/roles/695bad5e383d4052a3d6ffa680329cb7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:24.192283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71989eb-fb12-4e88-8dde-cbd67448130b None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:24.282652 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:24.330435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:24.379831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5617e-4ca6-4af1-98f1-457d0eb06880 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:24.401490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71989eb-fb12-4e88-8dde-cbd67448130b None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:24.428174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91ffdb-175b-4fdd-ad7d-6581529b39c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:24.428939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91ffdb-175b-4fdd-ad7d-6581529b39c6 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:24.429339 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91ffdb-175b-4fdd-ad7d-6581529b39c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:24.429713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91ffdb-175b-4fdd-ad7d-6581529b39c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:24.431025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91ffdb-175b-4fdd-ad7d-6581529b39c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:24.431438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91ffdb-175b-4fdd-ad7d-6581529b39c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:24.433462 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91ffdb-175b-4fdd-ad7d-6581529b39c6 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:24.445809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaa77b-c34c-4bda-8f1f-12e72374ad0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:24.446665 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaa77b-c34c-4bda-8f1f-12e72374ad0d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:24.447104 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaa77b-c34c-4bda-8f1f-12e72374ad0d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:24.447528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaa77b-c34c-4bda-8f1f-12e72374ad0d None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:24.448431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaa77b-c34c-4bda-8f1f-12e72374ad0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:24.448886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaa77b-c34c-4bda-8f1f-12e72374ad0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:24.449667 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 211/415] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:24.450990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaa77b-c34c-4bda-8f1f-12e72374ad0d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:24.456009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91ffdb-175b-4fdd-ad7d-6581529b39c6 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:24.474071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 205/416] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:24.482862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaa77b-c34c-4bda-8f1f-12e72374ad0d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:24.591006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91ffdb-175b-4fdd-ad7d-6581529b39c6 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:24.629180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab4567-1b8a-4bf0-8ca6-4cbebd1335a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:24.630234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab4567-1b8a-4bf0-8ca6-4cbebd1335a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:24.630677 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab4567-1b8a-4bf0-8ca6-4cbebd1335a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:24.631201 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab4567-1b8a-4bf0-8ca6-4cbebd1335a5 None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/286cfcbb8334421f87c0d426aa52a30a/roles/695bad5e383d4052a3d6ffa680329cb7` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:24.636270 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:24.639988 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaa77b-c34c-4bda-8f1f-12e72374ad0d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:24.645626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:24.649045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab4567-1b8a-4bf0-8ca6-4cbebd1335a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=286cfcbb8334421f87c0d426aa52a30a, role_id=695bad5e383d4052a3d6ffa680329cb7)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:24.650718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab4567-1b8a-4bf0-8ca6-4cbebd1335a5 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:24.674511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b519f-f167-43b6-ab45-47d935657077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:24.675164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b519f-f167-43b6-ab45-47d935657077 None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:24.675440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b519f-f167-43b6-ab45-47d935657077 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:24.675707 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b519f-f167-43b6-ab45-47d935657077 None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/b5ff8583997843258e315413abac172f/roles/695bad5e383d4052a3d6ffa680329cb7` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:24.691909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b519f-f167-43b6-ab45-47d935657077 None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=b5ff8583997843258e315413abac172f, role_id=695bad5e383d4052a3d6ffa680329cb7)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:24.693355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b519f-f167-43b6-ab45-47d935657077 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:24.751793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 212/417] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:12:24 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/286cfcbb8334421f87c0d426aa52a30a/roles/695bad5e383d4052a3d6ffa680329cb7 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:24.753332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306c8ce-2f6c-44d0-99bb-8724252fc0e4 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:24.753796 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306c8ce-2f6c-44d0-99bb-8724252fc0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:24.754089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306c8ce-2f6c-44d0-99bb-8724252fc0e4 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:24.755475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 213/418] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:24.761290 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564efb0b-ad3a-47f8-8356-05ed4374f95a None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:24.761574 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564efb0b-ad3a-47f8-8356-05ed4374f95a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:24.761846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564efb0b-ad3a-47f8-8356-05ed4374f95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:24.788636 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 206/419] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:12:24 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/b5ff8583997843258e315413abac172f/roles/695bad5e383d4052a3d6ffa680329cb7 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:24.791011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-564efb0b-ad3a-47f8-8356-05ed4374f95a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:24.792542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-564efb0b-ad3a-47f8-8356-05ed4374f95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:24.897365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 214/420] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:12:25.213471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b519f-f167-43b6-ab45-47d935657077 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:25.327801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b519f-f167-43b6-ab45-47d935657077 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:25.377107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd1ade-ffc8-467c-843b-d0b02c7dffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:25.378125 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd1ade-ffc8-467c-843b-d0b02c7dffa4 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:25.378632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd1ade-ffc8-467c-843b-d0b02c7dffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:25.379102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd1ade-ffc8-467c-843b-d0b02c7dffa4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:25.390435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfd1ade-ffc8-467c-843b-d0b02c7dffa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-fbfd1ade-ffc8-467c-843b-d0b02c7dffa4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:25.395651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:25.396515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:25.397286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:25.399533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 207/421] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:25.405885 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564efb0b-ad3a-47f8-8356-05ed4374f95a None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:25.558117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564efb0b-ad3a-47f8-8356-05ed4374f95a None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:25.582810 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:25.605452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f798686-3313-4719-af53-2c72adaac82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:25.606084 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f798686-3313-4719-af53-2c72adaac82a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:25.608475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f798686-3313-4719-af53-2c72adaac82a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:25.608475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f798686-3313-4719-af53-2c72adaac82a None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:25.608475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f798686-3313-4719-af53-2c72adaac82a None admin] RBAC: Adding query filter params (name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:25.608475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f798686-3313-4719-af53-2c72adaac82a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:25.609496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f798686-3313-4719-af53-2c72adaac82a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:25.621364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 215/422] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:25.627109 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd1ade-ffc8-467c-843b-d0b02c7dffa4 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:25.650802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:25.792244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfd1ade-ffc8-467c-843b-d0b02c7dffa4 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:25.853778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0556b-d569-4041-9106-fbc711fdd0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:25.854358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0556b-d569-4041-9106-fbc711fdd0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:25.854635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0556b-d569-4041-9106-fbc711fdd0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:25.854901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0556b-d569-4041-9106-fbc711fdd0f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:25.874108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0556b-d569-4041-9106-fbc711fdd0f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:25.875575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0556b-d569-4041-9106-fbc711fdd0f1 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-3db0556b-d569-4041-9106-fbc711fdd0f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:25.902921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:25.903877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:25.905255 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:25.905255 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:25.905255 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:25.905866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 208/423] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:25.910534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f798686-3313-4719-af53-2c72adaac82a None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:26.083158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f798686-3313-4719-af53-2c72adaac82a None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:26.108054 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a100f-d121-41b1-9e6d-1d4b1122b4a0 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:26.108280 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a100f-d121-41b1-9e6d-1d4b1122b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:26.108549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a100f-d121-41b1-9e6d-1d4b1122b4a0 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:26.110149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 209/424] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:26.115248 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f387772-4c46-4cdd-a473-f45131ae46eb None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:26.115532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f387772-4c46-4cdd-a473-f45131ae46eb None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:26.115803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f387772-4c46-4cdd-a473-f45131ae46eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:26.143080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b72f0-d3a6-4c74-88f0-1fc7fba5ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:26.143692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b72f0-d3a6-4c74-88f0-1fc7fba5ef12 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:26.143970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b72f0-d3a6-4c74-88f0-1fc7fba5ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:26.144242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b72f0-d3a6-4c74-88f0-1fc7fba5ef12 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:26.144883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b72f0-d3a6-4c74-88f0-1fc7fba5ef12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:26.145183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b72f0-d3a6-4c74-88f0-1fc7fba5ef12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:26.146583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b72f0-d3a6-4c74-88f0-1fc7fba5ef12 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:26.152228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-0f387772-4c46-4cdd-a473-f45131ae46eb None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:26.153410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0f387772-4c46-4cdd-a473-f45131ae46eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:26.167715 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 216/425] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:26.172841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b72f0-d3a6-4c74-88f0-1fc7fba5ef12 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:26.238296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:26.250428 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:26.260980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 210/426] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:26.350679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b72f0-d3a6-4c74-88f0-1fc7fba5ef12 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:26.397446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f0b93-9bbe-4289-829c-e88afcf2db08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:26.397992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f0b93-9bbe-4289-829c-e88afcf2db08 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:26.398295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f0b93-9bbe-4289-829c-e88afcf2db08 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:26.398570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f0b93-9bbe-4289-829c-e88afcf2db08 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:26.409428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f0b93-9bbe-4289-829c-e88afcf2db08 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:26.410891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f0b93-9bbe-4289-829c-e88afcf2db08 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-c65f0b93-9bbe-4289-829c-e88afcf2db08 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:26.425449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:26.426390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:26.427266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:26.427266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:26.427266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:26.427266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:26.427266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:26.427266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:26.427266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:26.427266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:26.427266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:12:26.427266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:26.427266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 217/427] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 19 02:12:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:26.432435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f387772-4c46-4cdd-a473-f45131ae46eb None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:26.546387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f387772-4c46-4cdd-a473-f45131ae46eb None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:26.580480 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c28d66-bdb8-4389-9bef-9cd7bdc36c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:26.581415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c28d66-bdb8-4389-9bef-9cd7bdc36c4a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:26.581891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c28d66-bdb8-4389-9bef-9cd7bdc36c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:26.582378 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c28d66-bdb8-4389-9bef-9cd7bdc36c4a None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:26.583292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c28d66-bdb8-4389-9bef-9cd7bdc36c4a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:26.583809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c28d66-bdb8-4389-9bef-9cd7bdc36c4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:26.585486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c28d66-bdb8-4389-9bef-9cd7bdc36c4a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:26.603076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 211/428] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 19 02:12:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:26.609258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f0b93-9bbe-4289-829c-e88afcf2db08 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:26.733789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:26.740792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f0b93-9bbe-4289-829c-e88afcf2db08 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:26.773911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c28d66-bdb8-4389-9bef-9cd7bdc36c4a None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:26.788855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03fc1a-20eb-45cb-a32f-33591f10acfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:26.789465 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03fc1a-20eb-45cb-a32f-33591f10acfb None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:26.789745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03fc1a-20eb-45cb-a32f-33591f10acfb None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:26.790016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03fc1a-20eb-45cb-a32f-33591f10acfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:26.841659 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:26.864382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03fc1a-20eb-45cb-a32f-33591f10acfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:26.865963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03fc1a-20eb-45cb-a32f-33591f10acfb None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:26.904326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c28d66-bdb8-4389-9bef-9cd7bdc36c4a None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:26.944597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1c03e-a215-41c4-9a11-779a1cb7254e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:26.945558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1c03e-a215-41c4-9a11-779a1cb7254e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:26.946030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1c03e-a215-41c4-9a11-779a1cb7254e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:26.946491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1c03e-a215-41c4-9a11-779a1cb7254e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:26.957981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1c03e-a215-41c4-9a11-779a1cb7254e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-61b1c03e-a215-41c4-9a11-779a1cb7254e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:26.963434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:26.964300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:26.965083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:26.973282 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 212/429] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:26.974374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:26.974920 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d253e-fe08-4867-b75f-12b0bc216f8e None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:26.975750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d253e-fe08-4867-b75f-12b0bc216f8e None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:26.976505 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76d253e-fe08-4867-b75f-12b0bc216f8e None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:26.978799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 213/430] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:26.979732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d253e-fe08-4867-b75f-12b0bc216f8e None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:26.983289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 218/431] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:12:26.989500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de9562-7b04-4d3c-ad41-37b4f26d9950 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:26.989500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de9562-7b04-4d3c-ad41-37b4f26d9950 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:26.989500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de9562-7b04-4d3c-ad41-37b4f26d9950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.006078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-13de9562-7b04-4d3c-ad41-37b4f26d9950 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:27.008490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-13de9562-7b04-4d3c-ad41-37b4f26d9950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:27.081949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 219/432] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:27.091218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de9562-7b04-4d3c-ad41-37b4f26d9950 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:27.094391 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76d253e-fe08-4867-b75f-12b0bc216f8e None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:27.112582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:27.139864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3178078-f807-4793-8a0a-b3f760211fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:27.142076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3178078-f807-4793-8a0a-b3f760211fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.142076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3178078-f807-4793-8a0a-b3f760211fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.142076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3178078-f807-4793-8a0a-b3f760211fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.142076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3178078-f807-4793-8a0a-b3f760211fc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:27.142659 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3178078-f807-4793-8a0a-b3f760211fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:27.147113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3178078-f807-4793-8a0a-b3f760211fc4 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:27.157181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 214/433] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:27.160126 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3178078-f807-4793-8a0a-b3f760211fc4 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:27.273056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3178078-f807-4793-8a0a-b3f760211fc4 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:27.298836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de9562-7b04-4d3c-ad41-37b4f26d9950 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:27.319101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d229d7-05c9-4753-bb00-22a6bc5a542d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:27.321345 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d229d7-05c9-4753-bb00-22a6bc5a542d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.323110 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d229d7-05c9-4753-bb00-22a6bc5a542d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.323629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d229d7-05c9-4753-bb00-22a6bc5a542d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.346913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d229d7-05c9-4753-bb00-22a6bc5a542d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:27.349875 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d229d7-05c9-4753-bb00-22a6bc5a542d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:27.350415 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:27.363355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ec0b2-4b6b-4655-9fb1-0ceac0c4c6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:27.368442 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ec0b2-4b6b-4655-9fb1-0ceac0c4c6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.368442 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ec0b2-4b6b-4655-9fb1-0ceac0c4c6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.368442 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ec0b2-4b6b-4655-9fb1-0ceac0c4c6c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.376813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ec0b2-4b6b-4655-9fb1-0ceac0c4c6c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:27.378368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ec0b2-4b6b-4655-9fb1-0ceac0c4c6c0 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-b8d229d7-05c9-4753-bb00-22a6bc5a542d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:27.378984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:27.379891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:27.380753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:27.380753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:27.380753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:27.383076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 215/434] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:27.384727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5912620-0a19-4052-826f-c0d9d2147fca None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.385303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5912620-0a19-4052-826f-c0d9d2147fca None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.385784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5912620-0a19-4052-826f-c0d9d2147fca None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.387221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 216/435] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:27.388149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5912620-0a19-4052-826f-c0d9d2147fca None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-ac2ec0b2-4b6b-4655-9fb1-0ceac0c4c6c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:27.405884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:27.406917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:27.407760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:27.407760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:27.407760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:27.410079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 220/436] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:27.411974 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece457a-fc1a-4f3e-9a91-0cfe33ead0ce None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.412256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece457a-fc1a-4f3e-9a91-0cfe33ead0ce None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.412529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece457a-fc1a-4f3e-9a91-0cfe33ead0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.415274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:27.434244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8ece457a-fc1a-4f3e-9a91-0cfe33ead0ce None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:27.435823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8ece457a-fc1a-4f3e-9a91-0cfe33ead0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:27.510103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 221/437] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:27.518068 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece457a-fc1a-4f3e-9a91-0cfe33ead0ce None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:27.518232 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5912620-0a19-4052-826f-c0d9d2147fca None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:27.563899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2324777-44c1-4817-b280-95b439ad92d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:27.564734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2324777-44c1-4817-b280-95b439ad92d6 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.565014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2324777-44c1-4817-b280-95b439ad92d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.565432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2324777-44c1-4817-b280-95b439ad92d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.566104 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2324777-44c1-4817-b280-95b439ad92d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:27.566412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2324777-44c1-4817-b280-95b439ad92d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:27.567768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2324777-44c1-4817-b280-95b439ad92d6 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:27.584689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 217/438] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:27.585528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2324777-44c1-4817-b280-95b439ad92d6 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:27.627280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece457a-fc1a-4f3e-9a91-0cfe33ead0ce None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:27.641228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:27.661802 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c64e2-f604-44cf-8241-e5b5d633f30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:27.662585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c64e2-f604-44cf-8241-e5b5d633f30f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.662934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c64e2-f604-44cf-8241-e5b5d633f30f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.663278 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c64e2-f604-44cf-8241-e5b5d633f30f None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.663991 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c64e2-f604-44cf-8241-e5b5d633f30f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:27.664375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c64e2-f604-44cf-8241-e5b5d633f30f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:27.665806 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c64e2-f604-44cf-8241-e5b5d633f30f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:27.671163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:27.676829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 222/439] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:27.680700 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65830020-165b-4ab5-ae55-6e74e2ca0ce8 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.682335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65830020-165b-4ab5-ae55-6e74e2ca0ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.682651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65830020-165b-4ab5-ae55-6e74e2ca0ce8 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.688539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 223/440] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:27.688539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65830020-165b-4ab5-ae55-6e74e2ca0ce8 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:27.697507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2324777-44c1-4817-b280-95b439ad92d6 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:27.730881 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8811e17-34bc-4704-ac93-2b3809caddc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:27.731500 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8811e17-34bc-4704-ac93-2b3809caddc9 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.731777 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8811e17-34bc-4704-ac93-2b3809caddc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.732047 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8811e17-34bc-4704-ac93-2b3809caddc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.741747 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8811e17-34bc-4704-ac93-2b3809caddc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-f8811e17-34bc-4704-ac93-2b3809caddc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:27.747040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:27.748079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:27.749521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 218/441] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:27.753884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8811e17-34bc-4704-ac93-2b3809caddc9 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:27.856596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8811e17-34bc-4704-ac93-2b3809caddc9 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:27.857535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65830020-165b-4ab5-ae55-6e74e2ca0ce8 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:27.891312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80075d20-25a6-42cf-a79f-25a89eabbef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:27.891499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3a842-6e07-4415-b905-a075bfc9b4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:27.891933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3a842-6e07-4415-b905-a075bfc9b4ec None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.892061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075d20-25a6-42cf-a79f-25a89eabbef4 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.892206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3a842-6e07-4415-b905-a075bfc9b4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.892314 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075d20-25a6-42cf-a79f-25a89eabbef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.892474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3a842-6e07-4415-b905-a075bfc9b4ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.892587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80075d20-25a6-42cf-a79f-25a89eabbef4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.899277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3a842-6e07-4415-b905-a075bfc9b4ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:27.899393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80075d20-25a6-42cf-a79f-25a89eabbef4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:27.900696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3a842-6e07-4415-b905-a075bfc9b4ec None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:27.900788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80075d20-25a6-42cf-a79f-25a89eabbef4 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-eda3a842-6e07-4415-b905-a075bfc9b4ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:27.912472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:27.913353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:27.914554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:27.914554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:27.914554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:27.914554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:27.914554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:27.914554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:27.914554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:27.914554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-80075d20-25a6-42cf-a79f-25a89eabbef4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:27.914940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:27.915718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:27.916579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:27.916579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:27.916579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:27.916579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:27.916579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:27.916579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:27.916579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:27.916579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:27.916579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 19 02:12:27.916579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:27.917088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 219/442] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:27.917088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367cfa6-619f-4f55-93dc-2b8ef5e5cbf2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:27.917088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367cfa6-619f-4f55-93dc-2b8ef5e5cbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:27.917088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367cfa6-619f-4f55-93dc-2b8ef5e5cbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:27.918483 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 224/443] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:12:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:27.919261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80075d20-25a6-42cf-a79f-25a89eabbef4 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:27.921633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:27.933375 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-f367cfa6-619f-4f55-93dc-2b8ef5e5cbf2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:27.935476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f367cfa6-619f-4f55-93dc-2b8ef5e5cbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:27.949121 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:28.000802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 220/444] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:28.001622 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367cfa6-619f-4f55-93dc-2b8ef5e5cbf2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.026126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80075d20-25a6-42cf-a79f-25a89eabbef4 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.060113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14779c60-691f-416c-9fb5-cf9611659d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.060766 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14779c60-691f-416c-9fb5-cf9611659d83 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.061101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14779c60-691f-416c-9fb5-cf9611659d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.061870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14779c60-691f-416c-9fb5-cf9611659d83 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.062675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14779c60-691f-416c-9fb5-cf9611659d83 None admin] RBAC: Adding query filter params (name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:28.063157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14779c60-691f-416c-9fb5-cf9611659d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:28.064601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14779c60-691f-416c-9fb5-cf9611659d83 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:28.077061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 225/445] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:28.077061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14779c60-691f-416c-9fb5-cf9611659d83 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.111279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367cfa6-619f-4f55-93dc-2b8ef5e5cbf2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.141778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:28.146968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f81d93-ed6e-4805-94de-1c5699605025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.147767 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f81d93-ed6e-4805-94de-1c5699605025 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.148045 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f81d93-ed6e-4805-94de-1c5699605025 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.148313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f81d93-ed6e-4805-94de-1c5699605025 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.158867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f81d93-ed6e-4805-94de-1c5699605025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-23f81d93-ed6e-4805-94de-1c5699605025 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:28.164058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:28.164918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:28.165692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 221/446] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:28.166848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f81d93-ed6e-4805-94de-1c5699605025 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.181275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:28.187941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14779c60-691f-416c-9fb5-cf9611659d83 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.222475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c2c44-8f59-4555-bd95-2c7fba869734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.223385 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c2c44-8f59-4555-bd95-2c7fba869734 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.223899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c2c44-8f59-4555-bd95-2c7fba869734 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.224403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c2c44-8f59-4555-bd95-2c7fba869734 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.225297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c2c44-8f59-4555-bd95-2c7fba869734 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:28.225751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c2c44-8f59-4555-bd95-2c7fba869734 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:28.227605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c2c44-8f59-4555-bd95-2c7fba869734 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:28.240825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 226/447] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:28.241782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c2c44-8f59-4555-bd95-2c7fba869734 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.270743 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f81d93-ed6e-4805-94de-1c5699605025 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.305994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1f2fc-cd48-4d24-8b30-7c82262eebd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.306621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1f2fc-cd48-4d24-8b30-7c82262eebd7 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.306901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1f2fc-cd48-4d24-8b30-7c82262eebd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.307170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1f2fc-cd48-4d24-8b30-7c82262eebd7 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.307895 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1f2fc-cd48-4d24-8b30-7c82262eebd7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:28.308201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1f2fc-cd48-4d24-8b30-7c82262eebd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:28.309616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1f2fc-cd48-4d24-8b30-7c82262eebd7 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:28.326058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 222/448] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:28.326923 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1f2fc-cd48-4d24-8b30-7c82262eebd7 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.349819 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c2c44-8f59-4555-bd95-2c7fba869734 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.377204 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:28.382313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d6403-5de3-4bb5-8ef0-851e2d77970c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.382933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d6403-5de3-4bb5-8ef0-851e2d77970c None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.383214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d6403-5de3-4bb5-8ef0-851e2d77970c None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.383485 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d6403-5de3-4bb5-8ef0-851e2d77970c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.393584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d6403-5de3-4bb5-8ef0-851e2d77970c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:28.395044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d6403-5de3-4bb5-8ef0-851e2d77970c None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-135d6403-5de3-4bb5-8ef0-851e2d77970c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:28.409398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:28.410807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:28.411690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:28.411690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:28.411690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:28.411690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 227/449] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:28.411690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d6403-5de3-4bb5-8ef0-851e2d77970c None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.421818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:28.429051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1f2fc-cd48-4d24-8b30-7c82262eebd7 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.461562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b0117-b03b-4087-816c-1a6544bf6109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.462192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0117-b03b-4087-816c-1a6544bf6109 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.462466 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0117-b03b-4087-816c-1a6544bf6109 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.462732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b0117-b03b-4087-816c-1a6544bf6109 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.463684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b0117-b03b-4087-816c-1a6544bf6109 None admin] RBAC: Adding query filter params (name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:28.463978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b0117-b03b-4087-816c-1a6544bf6109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:28.465339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b0117-b03b-4087-816c-1a6544bf6109 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:28.473584 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 223/450] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:28.474389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b0117-b03b-4087-816c-1a6544bf6109 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.510208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d6403-5de3-4bb5-8ef0-851e2d77970c None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.543719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279950be-0ee7-4e8d-b063-cdabd02019a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.544300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279950be-0ee7-4e8d-b063-cdabd02019a5 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.544573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279950be-0ee7-4e8d-b063-cdabd02019a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.544842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279950be-0ee7-4e8d-b063-cdabd02019a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.546100 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279950be-0ee7-4e8d-b063-cdabd02019a5 None admin] RBAC: Adding query filter params (role.id=695bad5e383d4052a3d6ffa680329cb7, scope.project.id=676e986ff972410d9f498800d313476f, user.id=0abf52399a344157ac4e0f9a8212164e) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:28.546413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279950be-0ee7-4e8d-b063-cdabd02019a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:28.547829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279950be-0ee7-4e8d-b063-cdabd02019a5 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:28.553302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 228/451] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/role_assignments?user.id=0abf52399a344157ac4e0f9a8212164e&scope.project.id=676e986ff972410d9f498800d313476f&role.id=695bad5e383d4052a3d6ffa680329cb7 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:28.554887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279950be-0ee7-4e8d-b063-cdabd02019a5 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.571918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b0117-b03b-4087-816c-1a6544bf6109 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.586779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:28.609786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c0bd8-1f1d-41ca-8a15-2e47992f4836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.610737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c0bd8-1f1d-41ca-8a15-2e47992f4836 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.611013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c0bd8-1f1d-41ca-8a15-2e47992f4836 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.611282 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c0bd8-1f1d-41ca-8a15-2e47992f4836 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.623483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c0bd8-1f1d-41ca-8a15-2e47992f4836 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-498c0bd8-1f1d-41ca-8a15-2e47992f4836 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:28.629808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:28.630790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:28.632703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 224/452] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:28.636644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c0bd8-1f1d-41ca-8a15-2e47992f4836 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.642234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:28.675077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279950be-0ee7-4e8d-b063-cdabd02019a5 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.711449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7818-318a-4f77-a267-b5b9c5858fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.712085 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7818-318a-4f77-a267-b5b9c5858fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.712362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7818-318a-4f77-a267-b5b9c5858fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.712633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7818-318a-4f77-a267-b5b9c5858fa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.725322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e7818-318a-4f77-a267-b5b9c5858fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:28.726820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e7818-318a-4f77-a267-b5b9c5858fa1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-c03e7818-318a-4f77-a267-b5b9c5858fa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:28.743095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:28.744023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:28.744840 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:28.744840 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:28.744840 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:28.744840 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 229/453] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:28.745124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7818-318a-4f77-a267-b5b9c5858fa1 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.803031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c0bd8-1f1d-41ca-8a15-2e47992f4836 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.837071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35619b1c-94db-4449-98e4-eb82de22ce82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.837758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35619b1c-94db-4449-98e4-eb82de22ce82 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.838383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35619b1c-94db-4449-98e4-eb82de22ce82 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.838651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35619b1c-94db-4449-98e4-eb82de22ce82 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.839284 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35619b1c-94db-4449-98e4-eb82de22ce82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:28.839584 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35619b1c-94db-4449-98e4-eb82de22ce82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:28.840925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35619b1c-94db-4449-98e4-eb82de22ce82 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:28.853239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 225/454] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:28.854034 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35619b1c-94db-4449-98e4-eb82de22ce82 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.867423 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7818-318a-4f77-a267-b5b9c5858fa1 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.888741 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:28.890246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:28.902493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f983b-224e-405e-b23f-4fa623438ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.904857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f983b-224e-405e-b23f-4fa623438ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.904857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f983b-224e-405e-b23f-4fa623438ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.904857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f983b-224e-405e-b23f-4fa623438ff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.914372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f983b-224e-405e-b23f-4fa623438ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:28.915849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934f983b-224e-405e-b23f-4fa623438ff9 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-934f983b-224e-405e-b23f-4fa623438ff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:28.931049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:28.931949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:28.932774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:28.932774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:28.932774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:28.938492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 230/455] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:28.938492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f983b-224e-405e-b23f-4fa623438ff9 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:28.958149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35619b1c-94db-4449-98e4-eb82de22ce82 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:28.995167 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b24aa2-99a8-400d-94dd-3301913a5250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:28.995772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b24aa2-99a8-400d-94dd-3301913a5250 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:28.996053 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b24aa2-99a8-400d-94dd-3301913a5250 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:28.996320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b24aa2-99a8-400d-94dd-3301913a5250 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:28.996978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b24aa2-99a8-400d-94dd-3301913a5250 None admin] RBAC: Adding query filter params (name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:28.997274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b24aa2-99a8-400d-94dd-3301913a5250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:28.999169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b24aa2-99a8-400d-94dd-3301913a5250 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:29.007309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 226/456] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:29.012946 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b24aa2-99a8-400d-94dd-3301913a5250 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:29.043181 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934f983b-224e-405e-b23f-4fa623438ff9 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:29.078915 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcf1d4-1814-4943-b3a5-dd1dc8dc5cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:29.079509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcf1d4-1814-4943-b3a5-dd1dc8dc5cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:29.079782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcf1d4-1814-4943-b3a5-dd1dc8dc5cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:29.080047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fcf1d4-1814-4943-b3a5-dd1dc8dc5cb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:29.080677 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcf1d4-1814-4943-b3a5-dd1dc8dc5cb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:29.080971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcf1d4-1814-4943-b3a5-dd1dc8dc5cb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:29.082355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fcf1d4-1814-4943-b3a5-dd1dc8dc5cb8 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:29.099013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 231/457] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:29.099013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcf1d4-1814-4943-b3a5-dd1dc8dc5cb8 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:29.122115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:29.130665 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b24aa2-99a8-400d-94dd-3301913a5250 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:29.134374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:29.165978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9948-46f5-49d9-8ab7-a9f9da057460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:29.166558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9948-46f5-49d9-8ab7-a9f9da057460 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:29.166830 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9948-46f5-49d9-8ab7-a9f9da057460 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:29.167102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9948-46f5-49d9-8ab7-a9f9da057460 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:29.175256 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9948-46f5-49d9-8ab7-a9f9da057460 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:29.176682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9948-46f5-49d9-8ab7-a9f9da057460 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-14ba9948-46f5-49d9-8ab7-a9f9da057460 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:29.187942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:29.188812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:29.189687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:29.189687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:29.189687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:29.189687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:29.189687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:29.189687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:29.189687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:29.189687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:29.189687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 02:12:29.189687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:29.189687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 227/458] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:29.191080 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9948-46f5-49d9-8ab7-a9f9da057460 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:29.209843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fcf1d4-1814-4943-b3a5-dd1dc8dc5cb8 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:29.246691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ddbd0-867b-4cdc-9155-3ad48395d0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:29.247298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ddbd0-867b-4cdc-9155-3ad48395d0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:29.247575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ddbd0-867b-4cdc-9155-3ad48395d0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:29.248135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ddbd0-867b-4cdc-9155-3ad48395d0c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:29.248989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ddbd0-867b-4cdc-9155-3ad48395d0c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:29.249300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ddbd0-867b-4cdc-9155-3ad48395d0c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:29.250686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ddbd0-867b-4cdc-9155-3ad48395d0c0 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:29.263554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 232/459] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:29.264189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ddbd0-867b-4cdc-9155-3ad48395d0c0 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:29.303758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9948-46f5-49d9-8ab7-a9f9da057460 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:29.339072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b527ef7-a19d-4811-8247-b650888413b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:29.340029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527ef7-a19d-4811-8247-b650888413b6 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:29.340497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527ef7-a19d-4811-8247-b650888413b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:29.340954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527ef7-a19d-4811-8247-b650888413b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:29.352083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b527ef7-a19d-4811-8247-b650888413b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:29.353534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b527ef7-a19d-4811-8247-b650888413b6 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:29.361318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:29.367624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ddbd0-867b-4cdc-9155-3ad48395d0c0 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-7b527ef7-a19d-4811-8247-b650888413b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:29.368761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:29.369607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:29.370453 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:29.370453 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:29.370453 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:29.375908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 228/460] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:29.376647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b527ef7-a19d-4811-8247-b650888413b6 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:29.385671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:29.402659 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b03aa8-24ed-48dd-9d5e-ec0d602f6086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:29.403277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b03aa8-24ed-48dd-9d5e-ec0d602f6086 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:29.403552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b03aa8-24ed-48dd-9d5e-ec0d602f6086 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:29.403915 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b03aa8-24ed-48dd-9d5e-ec0d602f6086 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:29.410619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b03aa8-24ed-48dd-9d5e-ec0d602f6086 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:29.412068 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b03aa8-24ed-48dd-9d5e-ec0d602f6086 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-08b03aa8-24ed-48dd-9d5e-ec0d602f6086 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:29.424524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:29.425366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:29.426270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:29.426270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:29.426270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:29.426270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:29.426270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:29.426270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:29.426270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:29.426270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:29.426270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 233/461] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:29.426772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b03aa8-24ed-48dd-9d5e-ec0d602f6086 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:29.490942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b527ef7-a19d-4811-8247-b650888413b6 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:29.525131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df9005-8617-4abe-913d-399ffa503547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:29.525728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df9005-8617-4abe-913d-399ffa503547 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:29.526004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df9005-8617-4abe-913d-399ffa503547 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:29.526296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df9005-8617-4abe-913d-399ffa503547 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:29.527013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df9005-8617-4abe-913d-399ffa503547 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:29.527317 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df9005-8617-4abe-913d-399ffa503547 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:29.528626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b03aa8-24ed-48dd-9d5e-ec0d602f6086 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:29.528738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df9005-8617-4abe-913d-399ffa503547 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:29.547033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 229/462] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Oct 19 02:12:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:29.550958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df9005-8617-4abe-913d-399ffa503547 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:29.565368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52878afd-5927-4b13-9c07-d43a7b1a48a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:29.565989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52878afd-5927-4b13-9c07-d43a7b1a48a2 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:29.566290 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52878afd-5927-4b13-9c07-d43a7b1a48a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:29.566554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52878afd-5927-4b13-9c07-d43a7b1a48a2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:29.573347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52878afd-5927-4b13-9c07-d43a7b1a48a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:29.574799 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52878afd-5927-4b13-9c07-d43a7b1a48a2 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-52878afd-5927-4b13-9c07-d43a7b1a48a2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:29.587093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:29.588256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 234/463] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:29.589142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52878afd-5927-4b13-9c07-d43a7b1a48a2 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:29.607090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:29.617725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:29.656792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df9005-8617-4abe-913d-399ffa503547 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:29.695483 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52878afd-5927-4b13-9c07-d43a7b1a48a2 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:29.696745 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b310d-cc0c-4996-906f-1806e188db2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:29.697508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b310d-cc0c-4996-906f-1806e188db2b None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:29.697789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b310d-cc0c-4996-906f-1806e188db2b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:29.698099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b310d-cc0c-4996-906f-1806e188db2b None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:29.699506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b310d-cc0c-4996-906f-1806e188db2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:29.699818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b310d-cc0c-4996-906f-1806e188db2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:29.701172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b310d-cc0c-4996-906f-1806e188db2b None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:29.714841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 230/464] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:29.715672 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b310d-cc0c-4996-906f-1806e188db2b None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:29.732095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb875278-ba13-4f54-9d8a-c9a2c86eab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:29.733481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb875278-ba13-4f54-9d8a-c9a2c86eab83 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:29.733931 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb875278-ba13-4f54-9d8a-c9a2c86eab83 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:29.734401 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb875278-ba13-4f54-9d8a-c9a2c86eab83 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:29.735220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb875278-ba13-4f54-9d8a-c9a2c86eab83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:29.735693 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb875278-ba13-4f54-9d8a-c9a2c86eab83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:29.737229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb875278-ba13-4f54-9d8a-c9a2c86eab83 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:29.749080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 235/465] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:29.750007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb875278-ba13-4f54-9d8a-c9a2c86eab83 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:29.829218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b310d-cc0c-4996-906f-1806e188db2b None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:29.830459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:29.854428 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:29.855306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb875278-ba13-4f54-9d8a-c9a2c86eab83 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:29.865065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20da2d0-dd01-45f6-b400-1bc8ca81e256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:29.865676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da2d0-dd01-45f6-b400-1bc8ca81e256 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:29.865949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da2d0-dd01-45f6-b400-1bc8ca81e256 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:29.866243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20da2d0-dd01-45f6-b400-1bc8ca81e256 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:29.894194 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be8671-a029-4f5c-a6fd-90d3c4842415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:29.895325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be8671-a029-4f5c-a6fd-90d3c4842415 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:29.895837 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be8671-a029-4f5c-a6fd-90d3c4842415 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:29.896342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be8671-a029-4f5c-a6fd-90d3c4842415 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:29.897490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be8671-a029-4f5c-a6fd-90d3c4842415 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:29.897924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be8671-a029-4f5c-a6fd-90d3c4842415 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:29.899536 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be8671-a029-4f5c-a6fd-90d3c4842415 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:29.916647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 236/466] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Oct 19 02:12:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:29.917454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be8671-a029-4f5c-a6fd-90d3c4842415 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:29.943827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20da2d0-dd01-45f6-b400-1bc8ca81e256 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:29.945405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20da2d0-dd01-45f6-b400-1bc8ca81e256 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:30.019775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be8671-a029-4f5c-a6fd-90d3c4842415 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:30.049676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:30.052014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 231/467] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:12:30.054577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1d76b-5124-41f7-8ee0-e0fa4329e29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:30.055244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1d76b-5124-41f7-8ee0-e0fa4329e29b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:30.055520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1d76b-5124-41f7-8ee0-e0fa4329e29b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:30.055812 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1d76b-5124-41f7-8ee0-e0fa4329e29b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:30.057036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20da2d0-dd01-45f6-b400-1bc8ca81e256 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:30.063439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1d76b-5124-41f7-8ee0-e0fa4329e29b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:30.064980 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1d76b-5124-41f7-8ee0-e0fa4329e29b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-d5e1d76b-5124-41f7-8ee0-e0fa4329e29b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:30.077648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:30.078607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 237/468] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:30.080027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1d76b-5124-41f7-8ee0-e0fa4329e29b None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:30.087239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:30.166093 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20da2d0-dd01-45f6-b400-1bc8ca81e256 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:30.200529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ea9be-9852-4ce3-ba21-884aae706378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:30.201184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ea9be-9852-4ce3-ba21-884aae706378 None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:30.201492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ea9be-9852-4ce3-ba21-884aae706378 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:30.202305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ea9be-9852-4ce3-ba21-884aae706378 None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/e476a32b458c4ea0bdc207d0ff4bfd47/roles/695bad5e383d4052a3d6ffa680329cb7` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:30.204062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1d76b-5124-41f7-8ee0-e0fa4329e29b None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:30.217323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ea9be-9852-4ce3-ba21-884aae706378 None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=e476a32b458c4ea0bdc207d0ff4bfd47, role_id=695bad5e383d4052a3d6ffa680329cb7)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:30.219695 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ea9be-9852-4ce3-ba21-884aae706378 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:30.239462 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b999c-834b-4fb0-bbc6-3adcbc5a03fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:30.240108 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b999c-834b-4fb0-bbc6-3adcbc5a03fa None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:30.240388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b999c-834b-4fb0-bbc6-3adcbc5a03fa None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:30.240659 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b999c-834b-4fb0-bbc6-3adcbc5a03fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:30.297507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:30.301901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 232/469] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:12:30 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/e476a32b458c4ea0bdc207d0ff4bfd47/roles/695bad5e383d4052a3d6ffa680329cb7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:30.301901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ea9be-9852-4ce3-ba21-884aae706378 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:30.308846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b999c-834b-4fb0-bbc6-3adcbc5a03fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:30.310333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b999c-834b-4fb0-bbc6-3adcbc5a03fa None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:30.414942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:30.445364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ea9be-9852-4ce3-ba21-884aae706378 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:30.447110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 238/470] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:12:30.451146 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b999c-834b-4fb0-bbc6-3adcbc5a03fa None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:30.491607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c5654-5a19-4be5-a283-4ed1b25e4c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:30.493462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c5654-5a19-4be5-a283-4ed1b25e4c8a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:30.493462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c5654-5a19-4be5-a283-4ed1b25e4c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:30.493462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c5654-5a19-4be5-a283-4ed1b25e4c8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:30.507531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c5654-5a19-4be5-a283-4ed1b25e4c8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:30.509360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c5654-5a19-4be5-a283-4ed1b25e4c8a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-171c5654-5a19-4be5-a283-4ed1b25e4c8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:30.527490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:30.528410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:30.529237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:30.529237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:30.529237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:30.529237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 233/471] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:30.529440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c5654-5a19-4be5-a283-4ed1b25e4c8a None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:30.580745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:30.586891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b999c-834b-4fb0-bbc6-3adcbc5a03fa None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:30.624606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bfddb-e2dd-4537-9b86-aec82239f03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:30.625857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bfddb-e2dd-4537-9b86-aec82239f03a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:30.626250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bfddb-e2dd-4537-9b86-aec82239f03a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:30.626840 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bfddb-e2dd-4537-9b86-aec82239f03a None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:30.627990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bfddb-e2dd-4537-9b86-aec82239f03a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:30.628814 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bfddb-e2dd-4537-9b86-aec82239f03a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:30.630446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bfddb-e2dd-4537-9b86-aec82239f03a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:30.651321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c5654-5a19-4be5-a283-4ed1b25e4c8a None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:30.653212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 239/472] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Oct 19 02:12:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:30.654430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bfddb-e2dd-4537-9b86-aec82239f03a None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:30.688555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:30.698881 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c966c0-b05b-47bc-8b3d-d195f7c70e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:30.699697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c966c0-b05b-47bc-8b3d-d195f7c70e6d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:30.700126 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c966c0-b05b-47bc-8b3d-d195f7c70e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:30.700392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c966c0-b05b-47bc-8b3d-d195f7c70e6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:30.712914 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c966c0-b05b-47bc-8b3d-d195f7c70e6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:30.715176 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c966c0-b05b-47bc-8b3d-d195f7c70e6d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-d1c966c0-b05b-47bc-8b3d-d195f7c70e6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:30.732612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:30.733850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:30.734867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:30.734867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:30.734867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:30.734867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 234/473] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:30.734867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c966c0-b05b-47bc-8b3d-d195f7c70e6d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:30.786398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bfddb-e2dd-4537-9b86-aec82239f03a None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:30.821251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e096d-57bb-489b-80f7-5817b09b3d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:30.821851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e096d-57bb-489b-80f7-5817b09b3d58 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:30.822154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e096d-57bb-489b-80f7-5817b09b3d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:30.822432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e096d-57bb-489b-80f7-5817b09b3d58 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:30.823073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e096d-57bb-489b-80f7-5817b09b3d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:30.823674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e096d-57bb-489b-80f7-5817b09b3d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:30.824736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e096d-57bb-489b-80f7-5817b09b3d58 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:30.837234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 240/474] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:30.838541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e096d-57bb-489b-80f7-5817b09b3d58 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:30.845956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c966c0-b05b-47bc-8b3d-d195f7c70e6d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:30.851343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:30.880692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694c8a3-39ed-4515-9c53-7f1f2cce7263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:30.881302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694c8a3-39ed-4515-9c53-7f1f2cce7263 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:30.881577 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694c8a3-39ed-4515-9c53-7f1f2cce7263 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:30.881847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694c8a3-39ed-4515-9c53-7f1f2cce7263 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:30.930661 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:30.942875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e096d-57bb-489b-80f7-5817b09b3d58 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:30.953358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694c8a3-39ed-4515-9c53-7f1f2cce7263 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:30.954915 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694c8a3-39ed-4515-9c53-7f1f2cce7263 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:30.981034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9464c-959f-4191-8417-6fd6962ef45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:30.981640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9464c-959f-4191-8417-6fd6962ef45d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:30.981916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9464c-959f-4191-8417-6fd6962ef45d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:30.982234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9464c-959f-4191-8417-6fd6962ef45d None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:30.983369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9464c-959f-4191-8417-6fd6962ef45d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:30.983679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9464c-959f-4191-8417-6fd6962ef45d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:30.985045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9464c-959f-4191-8417-6fd6962ef45d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:31.000993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 241/475] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:31.000993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b601f-f77a-4202-adfa-4ef941372f0b None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:31.000993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b601f-f77a-4202-adfa-4ef941372f0b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:31.000993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b601f-f77a-4202-adfa-4ef941372f0b None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:31.000993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 242/476] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:31.000993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b601f-f77a-4202-adfa-4ef941372f0b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:31.070026 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 235/477] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:12:31.070026 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694c8a3-39ed-4515-9c53-7f1f2cce7263 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:31.080131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:31.118453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b601f-f77a-4202-adfa-4ef941372f0b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:31.153757 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c34034-6f48-4a83-bcc7-a17625c92081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:31.154412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c34034-6f48-4a83-bcc7-a17625c92081 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:31.154686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c34034-6f48-4a83-bcc7-a17625c92081 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:31.155479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c34034-6f48-4a83-bcc7-a17625c92081 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:31.156321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:31.162009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c34034-6f48-4a83-bcc7-a17625c92081 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:31.163470 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c34034-6f48-4a83-bcc7-a17625c92081 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-95c34034-6f48-4a83-bcc7-a17625c92081 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:31.174823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:31.175644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:31.176612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:31.176612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:31.176612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:31.176612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:31.176612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:31.176612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:31.176612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:31.176612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:31.176612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 243/478] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:31.177091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a200c99-b387-45a5-8561-d6d1da72cbcf None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:31.177471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a200c99-b387-45a5-8561-d6d1da72cbcf None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:31.177562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a200c99-b387-45a5-8561-d6d1da72cbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:31.183218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694c8a3-39ed-4515-9c53-7f1f2cce7263 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:31.196253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-6a200c99-b387-45a5-8561-d6d1da72cbcf None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:31.197421 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6a200c99-b387-45a5-8561-d6d1da72cbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:31.221956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bac945-8838-404f-8daa-e2eb553b91a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:31.222616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bac945-8838-404f-8daa-e2eb553b91a0 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:31.222895 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bac945-8838-404f-8daa-e2eb553b91a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:31.223166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bac945-8838-404f-8daa-e2eb553b91a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:31.229852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bac945-8838-404f-8daa-e2eb553b91a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:31.231330 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bac945-8838-404f-8daa-e2eb553b91a0 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-57bac945-8838-404f-8daa-e2eb553b91a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:31.242517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:31.243424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:31.244303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:31.244303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:31.244303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:31.244303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:31.244303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:31.244303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:31.244303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:31.244303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:31.244303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 236/479] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:31.244765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bac945-8838-404f-8daa-e2eb553b91a0 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:31.272444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 244/480] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:31.273293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a200c99-b387-45a5-8561-d6d1da72cbcf None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:31.341097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:31.382490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a200c99-b387-45a5-8561-d6d1da72cbcf None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:31.385741 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bac945-8838-404f-8daa-e2eb553b91a0 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:31.416430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bc972-4fce-499a-87c3-bf53f260b2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:31.417082 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bc972-4fce-499a-87c3-bf53f260b2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:31.417358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bc972-4fce-499a-87c3-bf53f260b2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:31.417622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bc972-4fce-499a-87c3-bf53f260b2b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:31.418330 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bc972-4fce-499a-87c3-bf53f260b2b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:31.418633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bc972-4fce-499a-87c3-bf53f260b2b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:31.420047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bc972-4fce-499a-87c3-bf53f260b2b3 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:31.431309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95960fb5-f07d-453f-a394-86047aa86cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:31.432142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95960fb5-f07d-453f-a394-86047aa86cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:31.432582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95960fb5-f07d-453f-a394-86047aa86cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:31.433075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95960fb5-f07d-453f-a394-86047aa86cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:31.437630 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 245/481] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:31.438572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bc972-4fce-499a-87c3-bf53f260b2b3 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:31.441791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:31.451094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95960fb5-f07d-453f-a394-86047aa86cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:31.453489 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95960fb5-f07d-453f-a394-86047aa86cb3 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-95960fb5-f07d-453f-a394-86047aa86cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:31.475598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:31.476737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:31.477992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:31.477992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:31.477992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:31.488963 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 237/482] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:31.490504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95960fb5-f07d-453f-a394-86047aa86cb3 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:31.550349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bc972-4fce-499a-87c3-bf53f260b2b3 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:31.573710 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:31.590308 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa42b8-cc8b-4ad1-a72c-f24a16de785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:31.590917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa42b8-cc8b-4ad1-a72c-f24a16de785d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:31.591198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa42b8-cc8b-4ad1-a72c-f24a16de785d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:31.591479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa42b8-cc8b-4ad1-a72c-f24a16de785d None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:31.592167 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa42b8-cc8b-4ad1-a72c-f24a16de785d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:31.592531 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa42b8-cc8b-4ad1-a72c-f24a16de785d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:31.593982 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa42b8-cc8b-4ad1-a72c-f24a16de785d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:31.605572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 246/483] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:31.606495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa42b8-cc8b-4ad1-a72c-f24a16de785d None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:31.616897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95960fb5-f07d-453f-a394-86047aa86cb3 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:31.651869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70fbc2-5582-4213-85bc-b3e933b2783a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:31.652397 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70fbc2-5582-4213-85bc-b3e933b2783a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:31.652728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70fbc2-5582-4213-85bc-b3e933b2783a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:31.653052 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70fbc2-5582-4213-85bc-b3e933b2783a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:31.653781 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70fbc2-5582-4213-85bc-b3e933b2783a None admin] RBAC: Adding query filter params (role.id=695bad5e383d4052a3d6ffa680329cb7, scope.project.id=676e986ff972410d9f498800d313476f, user.id=e63c5bd8c4264db697947f025c67f512) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:31.654112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70fbc2-5582-4213-85bc-b3e933b2783a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:31.656123 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70fbc2-5582-4213-85bc-b3e933b2783a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:31.662198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 238/484] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:12:31 2022] GET /identity/v3/role_assignments?user.id=e63c5bd8c4264db697947f025c67f512&scope.project.id=676e986ff972410d9f498800d313476f&role.id=695bad5e383d4052a3d6ffa680329cb7 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:31.663063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70fbc2-5582-4213-85bc-b3e933b2783a None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:31.733583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa42b8-cc8b-4ad1-a72c-f24a16de785d None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:31.754469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:31.772321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec918d-facc-4b68-9bc6-c243801e82b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:31.773236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec918d-facc-4b68-9bc6-c243801e82b6 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:31.773554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec918d-facc-4b68-9bc6-c243801e82b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:31.773903 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec918d-facc-4b68-9bc6-c243801e82b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:31.774661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec918d-facc-4b68-9bc6-c243801e82b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:31.775031 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec918d-facc-4b68-9bc6-c243801e82b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:31.776513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec918d-facc-4b68-9bc6-c243801e82b6 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:31.795878 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 247/485] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:31.797050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec918d-facc-4b68-9bc6-c243801e82b6 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:31.805809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70fbc2-5582-4213-85bc-b3e933b2783a None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:31.845744 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:31.860543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2cb27-cf2f-4214-bfe4-e89195c3abcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:31.860913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2cb27-cf2f-4214-bfe4-e89195c3abcd None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:31.861267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2cb27-cf2f-4214-bfe4-e89195c3abcd None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:31.861605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2cb27-cf2f-4214-bfe4-e89195c3abcd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:31.872552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2cb27-cf2f-4214-bfe4-e89195c3abcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-f9f2cb27-cf2f-4214-bfe4-e89195c3abcd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:31.878551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:31.879421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:31.880205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 239/486] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:12:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:31.880943 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2cb27-cf2f-4214-bfe4-e89195c3abcd None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:31.932397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec918d-facc-4b68-9bc6-c243801e82b6 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:31.969612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66514c2-c800-49ee-bcf9-c569f43b7244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:31.970280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66514c2-c800-49ee-bcf9-c569f43b7244 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:31.971313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66514c2-c800-49ee-bcf9-c569f43b7244 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:31.971313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66514c2-c800-49ee-bcf9-c569f43b7244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:31.971613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66514c2-c800-49ee-bcf9-c569f43b7244 None admin] RBAC: Adding query filter params (role.id=695bad5e383d4052a3d6ffa680329cb7, scope.project.id=676e986ff972410d9f498800d313476f, user.id=286cfcbb8334421f87c0d426aa52a30a) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:31.971925 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66514c2-c800-49ee-bcf9-c569f43b7244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:31.973471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66514c2-c800-49ee-bcf9-c569f43b7244 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:31.980045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 248/487] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:12:31 2022] GET /identity/v3/role_assignments?user.id=286cfcbb8334421f87c0d426aa52a30a&scope.project.id=676e986ff972410d9f498800d313476f&role.id=695bad5e383d4052a3d6ffa680329cb7 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:31.980955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66514c2-c800-49ee-bcf9-c569f43b7244 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:32.009108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2cb27-cf2f-4214-bfe4-e89195c3abcd None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:32.044860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:32.086515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bdd54-312a-4e72-aea9-3aab4b7cec39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:32.087385 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bdd54-312a-4e72-aea9-3aab4b7cec39 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:32.087766 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bdd54-312a-4e72-aea9-3aab4b7cec39 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:32.088539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bdd54-312a-4e72-aea9-3aab4b7cec39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:32.106713 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bdd54-312a-4e72-aea9-3aab4b7cec39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:32.112780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bdd54-312a-4e72-aea9-3aab4b7cec39 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-4e6bdd54-312a-4e72-aea9-3aab4b7cec39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:32.132459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:32.133558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:32.134644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:32.134644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:32.134644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:32.134644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:32.134644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:32.134644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:32.134644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:32.134644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:32.138238 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 240/488] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:32.143845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bdd54-312a-4e72-aea9-3aab4b7cec39 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:32.192229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66514c2-c800-49ee-bcf9-c569f43b7244 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:32.198351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:32.233718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176e5eb-b34b-45d1-9fe1-b8fee6565716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:32.235044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176e5eb-b34b-45d1-9fe1-b8fee6565716 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:32.235044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176e5eb-b34b-45d1-9fe1-b8fee6565716 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:32.236024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176e5eb-b34b-45d1-9fe1-b8fee6565716 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:32.237511 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176e5eb-b34b-45d1-9fe1-b8fee6565716 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:32.237929 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176e5eb-b34b-45d1-9fe1-b8fee6565716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:32.239432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176e5eb-b34b-45d1-9fe1-b8fee6565716 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:32.248342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 249/489] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:32.254085 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176e5eb-b34b-45d1-9fe1-b8fee6565716 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:32.266081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bdd54-312a-4e72-aea9-3aab4b7cec39 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:32.311166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16848-c936-43bb-a082-27e68e9553ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:32.312035 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16848-c936-43bb-a082-27e68e9553ac None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:32.312462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16848-c936-43bb-a082-27e68e9553ac None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:32.312876 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16848-c936-43bb-a082-27e68e9553ac None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:32.313729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d16848-c936-43bb-a082-27e68e9553ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:32.314205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d16848-c936-43bb-a082-27e68e9553ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:32.316315 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d16848-c936-43bb-a082-27e68e9553ac None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:32.337024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 241/490] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:32.344130 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16848-c936-43bb-a082-27e68e9553ac None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:32.375662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176e5eb-b34b-45d1-9fe1-b8fee6565716 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:32.380241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:32.425874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca734b-c901-4d72-9392-9b710f4a6121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:32.426784 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca734b-c901-4d72-9392-9b710f4a6121 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:32.427668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca734b-c901-4d72-9392-9b710f4a6121 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:32.428093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca734b-c901-4d72-9392-9b710f4a6121 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:32.443062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca734b-c901-4d72-9392-9b710f4a6121 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:32.445426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eca734b-c901-4d72-9392-9b710f4a6121 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:32.453070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16848-c936-43bb-a082-27e68e9553ac None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-5eca734b-c901-4d72-9392-9b710f4a6121 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:32.467789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:32.469516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:32.470836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:32.470836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:32.470836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:32.473234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 250/491] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:32.479078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca734b-c901-4d72-9392-9b710f4a6121 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:32.504084 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:32.509970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a9419-cd91-4402-bab0-a8746573af2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:32.510891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9419-cd91-4402-bab0-a8746573af2c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:32.511317 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9419-cd91-4402-bab0-a8746573af2c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:32.511764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9419-cd91-4402-bab0-a8746573af2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:32.512688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a9419-cd91-4402-bab0-a8746573af2c None admin] RBAC: Adding query filter params (role.id=695bad5e383d4052a3d6ffa680329cb7, scope.project.id=676e986ff972410d9f498800d313476f, user.id=b5ff8583997843258e315413abac172f) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:32.513833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a9419-cd91-4402-bab0-a8746573af2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:32.516800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a9419-cd91-4402-bab0-a8746573af2c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:32.530090 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 242/492] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:12:32 2022] GET /identity/v3/role_assignments?user.id=b5ff8583997843258e315413abac172f&scope.project.id=676e986ff972410d9f498800d313476f&role.id=695bad5e383d4052a3d6ffa680329cb7 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:32.610372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028a646-49eb-431c-8991-266f84d8aa4d None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:32.610659 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028a646-49eb-431c-8991-266f84d8aa4d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:32.610933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028a646-49eb-431c-8991-266f84d8aa4d None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:32.612493 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 243/493] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:12:32.618686 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826fb1d-414c-4681-bc8e-5db853ccd12a None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:32.618966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826fb1d-414c-4681-bc8e-5db853ccd12a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:32.619243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826fb1d-414c-4681-bc8e-5db853ccd12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:32.627714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca734b-c901-4d72-9392-9b710f4a6121 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:32.642799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-1826fb1d-414c-4681-bc8e-5db853ccd12a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:32.644619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1826fb1d-414c-4681-bc8e-5db853ccd12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:32.666271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756bd27-b0bd-4ef1-949e-9beba520c82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:32.666879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756bd27-b0bd-4ef1-949e-9beba520c82e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:32.667153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756bd27-b0bd-4ef1-949e-9beba520c82e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:32.667521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756bd27-b0bd-4ef1-949e-9beba520c82e None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:32.668170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756bd27-b0bd-4ef1-949e-9beba520c82e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:32.668468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756bd27-b0bd-4ef1-949e-9beba520c82e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:32.669853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756bd27-b0bd-4ef1-949e-9beba520c82e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:32.683263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 251/494] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:32.691099 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756bd27-b0bd-4ef1-949e-9beba520c82e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:32.760554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 244/495] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:12:32.813906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:32.835781 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756bd27-b0bd-4ef1-949e-9beba520c82e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:32.870994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f03cb6-0334-49a7-8813-06b75b237496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:32.871624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f03cb6-0334-49a7-8813-06b75b237496 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:32.871902 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f03cb6-0334-49a7-8813-06b75b237496 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:32.872174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f03cb6-0334-49a7-8813-06b75b237496 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:32.880549 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f03cb6-0334-49a7-8813-06b75b237496 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:32.881990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f03cb6-0334-49a7-8813-06b75b237496 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-b0f03cb6-0334-49a7-8813-06b75b237496 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:32.893567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:32.894853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:32.896171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:32.896171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:32.896171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:32.896171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:32.896171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:32.896171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:32.896171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:32.896171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:32.896171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:12:32.896171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:32.896171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 252/496] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 19 02:12:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:32.900240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826fb1d-414c-4681-bc8e-5db853ccd12a None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:32.928977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:33.005715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826fb1d-414c-4681-bc8e-5db853ccd12a None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:33.040725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb41f55-c26c-4c87-88aa-ff310c6f1e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:33.041385 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb41f55-c26c-4c87-88aa-ff310c6f1e63 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:33.041632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb41f55-c26c-4c87-88aa-ff310c6f1e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:33.041901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb41f55-c26c-4c87-88aa-ff310c6f1e63 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:33.042657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb41f55-c26c-4c87-88aa-ff310c6f1e63 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:33.042961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb41f55-c26c-4c87-88aa-ff310c6f1e63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:33.044376 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb41f55-c26c-4c87-88aa-ff310c6f1e63 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:33.074760 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 245/497] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 19 02:12:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:33.077807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f03cb6-0334-49a7-8813-06b75b237496 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:33.080319 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb41f55-c26c-4c87-88aa-ff310c6f1e63 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:33.233877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f03cb6-0334-49a7-8813-06b75b237496 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:33.244896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:33.273244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb41f55-c26c-4c87-88aa-ff310c6f1e63 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:33.305903 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080f0a4-844c-45e7-8e62-1ca78aca1426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:33.306760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080f0a4-844c-45e7-8e62-1ca78aca1426 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:33.307139 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080f0a4-844c-45e7-8e62-1ca78aca1426 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:33.307511 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080f0a4-844c-45e7-8e62-1ca78aca1426 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:33.320030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:33.328894 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080f0a4-844c-45e7-8e62-1ca78aca1426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:33.334378 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d452d4a-eae1-4ba9-a389-d47a711f6c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:33.335140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d452d4a-eae1-4ba9-a389-d47a711f6c85 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:33.335533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d452d4a-eae1-4ba9-a389-d47a711f6c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:33.335899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d452d4a-eae1-4ba9-a389-d47a711f6c85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-d080f0a4-844c-45e7-8e62-1ca78aca1426 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:33.338738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:33.339829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:33.340762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:33.352714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 253/498] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:33.353959 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080f0a4-844c-45e7-8e62-1ca78aca1426 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:33.361954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d452d4a-eae1-4ba9-a389-d47a711f6c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:33.368502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d452d4a-eae1-4ba9-a389-d47a711f6c85 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-4d452d4a-eae1-4ba9-a389-d47a711f6c85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:33.385956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:33.386979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:33.388259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:33.388259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:33.388259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:33.388259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 246/499] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:33.393359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d452d4a-eae1-4ba9-a389-d47a711f6c85 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:33.525284 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080f0a4-844c-45e7-8e62-1ca78aca1426 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:33.578370 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e838001-864b-4fb6-95ae-90c2ae79381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:33.578891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e838001-864b-4fb6-95ae-90c2ae79381f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:33.580050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e838001-864b-4fb6-95ae-90c2ae79381f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:33.581376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e838001-864b-4fb6-95ae-90c2ae79381f None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:33.582070 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e838001-864b-4fb6-95ae-90c2ae79381f None admin] RBAC: Adding query filter params (name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:33.582368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e838001-864b-4fb6-95ae-90c2ae79381f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:33.583750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e838001-864b-4fb6-95ae-90c2ae79381f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:33.589495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d452d4a-eae1-4ba9-a389-d47a711f6c85 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:33.601777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 254/500] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:33.607644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e838001-864b-4fb6-95ae-90c2ae79381f None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:33.654696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eb170-8615-478a-884b-b2dfa5ff6572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:33.655843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb170-8615-478a-884b-b2dfa5ff6572 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:33.656119 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb170-8615-478a-884b-b2dfa5ff6572 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:33.656390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb170-8615-478a-884b-b2dfa5ff6572 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:33.657045 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb170-8615-478a-884b-b2dfa5ff6572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:33.657345 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb170-8615-478a-884b-b2dfa5ff6572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:33.659545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb170-8615-478a-884b-b2dfa5ff6572 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:33.672036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:33.681386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 247/501] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:33.687597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eb170-8615-478a-884b-b2dfa5ff6572 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:33.721873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:33.793080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e838001-864b-4fb6-95ae-90c2ae79381f None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:33.860338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010b72c-7ed7-4ae0-9c0a-1d4b6a88d1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:33.860900 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010b72c-7ed7-4ae0-9c0a-1d4b6a88d1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:33.861175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010b72c-7ed7-4ae0-9c0a-1d4b6a88d1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:33.861441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010b72c-7ed7-4ae0-9c0a-1d4b6a88d1d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:33.881633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eb170-8615-478a-884b-b2dfa5ff6572 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:33.886293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010b72c-7ed7-4ae0-9c0a-1d4b6a88d1d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:33.887758 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010b72c-7ed7-4ae0-9c0a-1d4b6a88d1d5 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-c010b72c-7ed7-4ae0-9c0a-1d4b6a88d1d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:33.920937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:33.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:33.922924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:33.922924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:33.922924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:33.922924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 255/502] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:33.922924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010b72c-7ed7-4ae0-9c0a-1d4b6a88d1d5 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:33.934755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2d0d4-0e95-4121-9898-7e9adb4be1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:33.935383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2d0d4-0e95-4121-9898-7e9adb4be1cb None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:33.935658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2d0d4-0e95-4121-9898-7e9adb4be1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:33.935929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2d0d4-0e95-4121-9898-7e9adb4be1cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:33.943313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2d0d4-0e95-4121-9898-7e9adb4be1cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:33.944874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c2d0d4-0e95-4121-9898-7e9adb4be1cb None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-82c2d0d4-0e95-4121-9898-7e9adb4be1cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:33.957025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:33.957873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:33.958768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:33.958768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:33.958768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:33.958768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:33.958768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:33.958768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:33.958768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:33.958768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:33.958768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 248/503] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:33.964307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2d0d4-0e95-4121-9898-7e9adb4be1cb None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:34.051232 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:34.093493 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2d0d4-0e95-4121-9898-7e9adb4be1cb None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:34.128260 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92521e2d-7280-4360-9afb-7313d9cb4579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:34.128791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92521e2d-7280-4360-9afb-7313d9cb4579 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:34.129063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92521e2d-7280-4360-9afb-7313d9cb4579 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:34.129326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92521e2d-7280-4360-9afb-7313d9cb4579 None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:34.130005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92521e2d-7280-4360-9afb-7313d9cb4579 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:34.130336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92521e2d-7280-4360-9afb-7313d9cb4579 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:34.131810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92521e2d-7280-4360-9afb-7313d9cb4579 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:34.144233 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 249/504] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:34.153136 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92521e2d-7280-4360-9afb-7313d9cb4579 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:34.175699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010b72c-7ed7-4ae0-9c0a-1d4b6a88d1d5 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:34.203850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:34.246251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9b4c7-3b7e-4cbd-b758-506694b6bb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:34.246855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9b4c7-3b7e-4cbd-b758-506694b6bb72 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:34.247131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9b4c7-3b7e-4cbd-b758-506694b6bb72 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:34.247401 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9b4c7-3b7e-4cbd-b758-506694b6bb72 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:34.249479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9b4c7-3b7e-4cbd-b758-506694b6bb72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:34.249781 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9b4c7-3b7e-4cbd-b758-506694b6bb72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:34.251178 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9b4c7-3b7e-4cbd-b758-506694b6bb72 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:34.278564 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 256/505] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:34.282303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0e77b-260e-4719-bb14-b72e6daa2e76 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:34.282585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0e77b-260e-4719-bb14-b72e6daa2e76 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:34.282855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0e77b-260e-4719-bb14-b72e6daa2e76 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:34.284397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 257/506] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:34.287272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0e77b-260e-4719-bb14-b72e6daa2e76 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:34.321992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92521e2d-7280-4360-9afb-7313d9cb4579 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:34.341395 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:34.361714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f866c-8d6f-44ae-a555-71c5eec9825f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:34.362891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f866c-8d6f-44ae-a555-71c5eec9825f None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:34.363384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f866c-8d6f-44ae-a555-71c5eec9825f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:34.363852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f866c-8d6f-44ae-a555-71c5eec9825f None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/0abf52399a344157ac4e0f9a8212164e/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:34.366169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f866c-8d6f-44ae-a555-71c5eec9825f None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=0abf52399a344157ac4e0f9a8212164e, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:34.367803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f866c-8d6f-44ae-a555-71c5eec9825f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:34.404712 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0e77b-260e-4719-bb14-b72e6daa2e76 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:34.440024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf3df9-4f27-404a-ae03-e117a73e15e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:34.440814 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf3df9-4f27-404a-ae03-e117a73e15e6 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:34.441098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf3df9-4f27-404a-ae03-e117a73e15e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:34.441371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf3df9-4f27-404a-ae03-e117a73e15e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:34.450994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf3df9-4f27-404a-ae03-e117a73e15e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:34.452613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf3df9-4f27-404a-ae03-e117a73e15e6 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:34.458365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 250/507] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:12:34 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/0abf52399a344157ac4e0f9a8212164e/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:12:34.460141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa5bf3-f8e5-4b5d-9440-d895388a3e75 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:34.460811 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa5bf3-f8e5-4b5d-9440-d895388a3e75 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:34.461287 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa5bf3-f8e5-4b5d-9440-d895388a3e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-4acf3df9-4f27-404a-ae03-e117a73e15e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:34.466348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:34.467828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:34.468752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:34.468752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:34.468752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:34.468752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:34.468752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:34.468752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:34.468752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:34.468752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:34.468752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 19 02:12:34.468752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:34.468752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 258/508] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:12:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:34.479695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf3df9-4f27-404a-ae03-e117a73e15e6 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:34.493911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-92fa5bf3-f8e5-4b5d-9440-d895388a3e75 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:34.496441 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-92fa5bf3-f8e5-4b5d-9440-d895388a3e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:34.551648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:34.583978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 251/509] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:34.697186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf3df9-4f27-404a-ae03-e117a73e15e6 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:34.738662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec2239-4b90-46d8-b108-cba6213f07af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:34.739296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec2239-4b90-46d8-b108-cba6213f07af None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:34.739538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec2239-4b90-46d8-b108-cba6213f07af None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:34.739813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec2239-4b90-46d8-b108-cba6213f07af None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:34.740551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec2239-4b90-46d8-b108-cba6213f07af None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:34.740859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec2239-4b90-46d8-b108-cba6213f07af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:34.742306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec2239-4b90-46d8-b108-cba6213f07af None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:34.760012 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 259/510] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 19 02:12:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:34.766258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa5bf3-f8e5-4b5d-9440-d895388a3e75 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:34.791700 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a02f1d-dd5d-4089-80ed-56c9ede8f3cb None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:34.792029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a02f1d-dd5d-4089-80ed-56c9ede8f3cb None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:34.792252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a02f1d-dd5d-4089-80ed-56c9ede8f3cb None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:34.793755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 260/511] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:34.799656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52ad1e-70f2-47bc-b535-fa2afaabb67d None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:34.799932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52ad1e-70f2-47bc-b535-fa2afaabb67d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:34.800200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52ad1e-70f2-47bc-b535-fa2afaabb67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:34.824562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-ba52ad1e-70f2-47bc-b535-fa2afaabb67d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:34.825864 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ba52ad1e-70f2-47bc-b535-fa2afaabb67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:34.914096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa5bf3-f8e5-4b5d-9440-d895388a3e75 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:34.924128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:34.953218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2bf61-19ec-4f4a-92eb-446ffbe6d008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:34.954259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2bf61-19ec-4f4a-92eb-446ffbe6d008 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:34.955009 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2bf61-19ec-4f4a-92eb-446ffbe6d008 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:34.955693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2bf61-19ec-4f4a-92eb-446ffbe6d008 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:34.966220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 261/512] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:12:34.967074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52ad1e-70f2-47bc-b535-fa2afaabb67d None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:35.026507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:35.032801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2bf61-19ec-4f4a-92eb-446ffbe6d008 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:35.034462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2bf61-19ec-4f4a-92eb-446ffbe6d008 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:35.149301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 252/513] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:12:35.158228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2bf61-19ec-4f4a-92eb-446ffbe6d008 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:35.171927 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:35.209602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52ad1e-70f2-47bc-b535-fa2afaabb67d None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:35.247580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b080b-3536-4ebf-9430-149f56c10a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:35.248458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b080b-3536-4ebf-9430-149f56c10a30 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:35.249481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b080b-3536-4ebf-9430-149f56c10a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:35.249911 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b080b-3536-4ebf-9430-149f56c10a30 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:35.250654 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b080b-3536-4ebf-9430-149f56c10a30 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:35.252579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b080b-3536-4ebf-9430-149f56c10a30 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-552b080b-3536-4ebf-9430-149f56c10a30 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:35.264452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:35.265496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 02:12:35.266358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:35.269078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 262/514] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Oct 19 02:12:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:35.270065 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b080b-3536-4ebf-9430-149f56c10a30 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:35.349763 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2bf61-19ec-4f4a-92eb-446ffbe6d008 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:35.361080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:35.377255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b080b-3536-4ebf-9430-149f56c10a30 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:35.386307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89884eb9-3866-4de2-bb9c-be56da9f7e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:35.386995 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89884eb9-3866-4de2-bb9c-be56da9f7e0f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:35.387210 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89884eb9-3866-4de2-bb9c-be56da9f7e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:35.387474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89884eb9-3866-4de2-bb9c-be56da9f7e0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:35.398854 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89884eb9-3866-4de2-bb9c-be56da9f7e0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:35.400300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89884eb9-3866-4de2-bb9c-be56da9f7e0f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:35.413975 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175110a8-6028-4bf2-a2f2-79c926e3af39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:35.414629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175110a8-6028-4bf2-a2f2-79c926e3af39 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:35.414910 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175110a8-6028-4bf2-a2f2-79c926e3af39 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:35.415182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175110a8-6028-4bf2-a2f2-79c926e3af39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-89884eb9-3866-4de2-bb9c-be56da9f7e0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:35.415515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:35.416467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:35.417333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:35.417333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:35.417333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:35.417333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 253/515] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:35.418144 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1eb56-61d2-4023-9ad9-d9720c6419ee None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:35.418144 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1eb56-61d2-4023-9ad9-d9720c6419ee None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:35.418285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1eb56-61d2-4023-9ad9-d9720c6419ee None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:35.419648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 254/516] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:35.420147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1eb56-61d2-4023-9ad9-d9720c6419ee None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:35.435075 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175110a8-6028-4bf2-a2f2-79c926e3af39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:35.436578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175110a8-6028-4bf2-a2f2-79c926e3af39 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-175110a8-6028-4bf2-a2f2-79c926e3af39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:35.452149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:35.453113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:35.455105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:35.455105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:35.455105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:35.455105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 263/517] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:35.455105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175110a8-6028-4bf2-a2f2-79c926e3af39 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:35.492164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:35.535813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1eb56-61d2-4023-9ad9-d9720c6419ee None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:35.563395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175110a8-6028-4bf2-a2f2-79c926e3af39 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:35.577786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924ec8f-3e57-4ed8-adb0-f4b335578213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:35.578439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924ec8f-3e57-4ed8-adb0-f4b335578213 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:35.578722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924ec8f-3e57-4ed8-adb0-f4b335578213 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:35.578994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924ec8f-3e57-4ed8-adb0-f4b335578213 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:35.579632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924ec8f-3e57-4ed8-adb0-f4b335578213 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:35.579935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924ec8f-3e57-4ed8-adb0-f4b335578213 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:35.581296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924ec8f-3e57-4ed8-adb0-f4b335578213 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:35.585561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 255/518] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 19 02:12:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:35.590343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007802f-c9cd-4a9a-bb89-b434f3a237de None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:35.590343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007802f-c9cd-4a9a-bb89-b434f3a237de None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:35.590343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007802f-c9cd-4a9a-bb89-b434f3a237de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:35.605252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-3007802f-c9cd-4a9a-bb89-b434f3a237de None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:35.605551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a676a-1ba8-4763-9993-80a81fc8ee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:35.606182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a676a-1ba8-4763-9993-80a81fc8ee51 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:35.606464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a676a-1ba8-4763-9993-80a81fc8ee51 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:35.606639 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3007802f-c9cd-4a9a-bb89-b434f3a237de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:35.606732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a676a-1ba8-4763-9993-80a81fc8ee51 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:35.607358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a676a-1ba8-4763-9993-80a81fc8ee51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:35.607714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a676a-1ba8-4763-9993-80a81fc8ee51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:35.610722 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a676a-1ba8-4763-9993-80a81fc8ee51 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:35.613477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:35.628061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 264/519] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:35.629519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a676a-1ba8-4763-9993-80a81fc8ee51 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:35.686668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 256/520] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:35.687828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007802f-c9cd-4a9a-bb89-b434f3a237de None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:35.732620 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a676a-1ba8-4763-9993-80a81fc8ee51 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:35.760811 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:35.780516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b39460-7e02-44ec-8add-f85c663c50ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:35.782089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39460-7e02-44ec-8add-f85c663c50ca None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:35.782889 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39460-7e02-44ec-8add-f85c663c50ca None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:35.783163 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39460-7e02-44ec-8add-f85c663c50ca None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:35.783851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b39460-7e02-44ec-8add-f85c663c50ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:35.784152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b39460-7e02-44ec-8add-f85c663c50ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:35.785645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b39460-7e02-44ec-8add-f85c663c50ca None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:35.804481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 265/521] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:35.804481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b39460-7e02-44ec-8add-f85c663c50ca None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:35.819253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3007802f-c9cd-4a9a-bb89-b434f3a237de None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:35.856061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003b10d-4401-4afb-9a8b-ff091f7fcc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:35.857046 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003b10d-4401-4afb-9a8b-ff091f7fcc9c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:35.857528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003b10d-4401-4afb-9a8b-ff091f7fcc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:35.858081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003b10d-4401-4afb-9a8b-ff091f7fcc9c None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:35.858940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003b10d-4401-4afb-9a8b-ff091f7fcc9c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:35.859444 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003b10d-4401-4afb-9a8b-ff091f7fcc9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:35.861063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003b10d-4401-4afb-9a8b-ff091f7fcc9c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:35.867414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 257/522] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 19 02:12:35 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:35.868494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003b10d-4401-4afb-9a8b-ff091f7fcc9c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:35.886027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:35.972306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b39460-7e02-44ec-8add-f85c663c50ca None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:36.019935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c324fb8-b203-48cc-b853-dd0e5462ae56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:36.021636 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c324fb8-b203-48cc-b853-dd0e5462ae56 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:36.022591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c324fb8-b203-48cc-b853-dd0e5462ae56 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:36.023095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c324fb8-b203-48cc-b853-dd0e5462ae56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:36.024169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003b10d-4401-4afb-9a8b-ff091f7fcc9c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:36.033629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c324fb8-b203-48cc-b853-dd0e5462ae56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:36.036064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c324fb8-b203-48cc-b853-dd0e5462ae56 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:36.044555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-4c324fb8-b203-48cc-b853-dd0e5462ae56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:36.056609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:36.057657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:36.058611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:36.058611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:36.058611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:36.058611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:36.058611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:36.058611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:36.058611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:36.058611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:36.060612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 266/523] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:36.061738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c324fb8-b203-48cc-b853-dd0e5462ae56 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:36.073226 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f255ba-96c8-4022-95d7-aa36c0f0bd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:36.073983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f255ba-96c8-4022-95d7-aa36c0f0bd33 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:36.074390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f255ba-96c8-4022-95d7-aa36c0f0bd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:36.074819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f255ba-96c8-4022-95d7-aa36c0f0bd33 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:36.075610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f255ba-96c8-4022-95d7-aa36c0f0bd33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:36.077813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f255ba-96c8-4022-95d7-aa36c0f0bd33 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:36.169834 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 258/524] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:12:35 2022] POST /identity/v3/users => generated 254 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:36.171249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f255ba-96c8-4022-95d7-aa36c0f0bd33 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:36.198629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c324fb8-b203-48cc-b853-dd0e5462ae56 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:36.234239 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:36.273601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9deb7-f147-40c4-a3e7-c3d1fea790ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:36.274185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9deb7-f147-40c4-a3e7-c3d1fea790ef None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:36.274467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9deb7-f147-40c4-a3e7-c3d1fea790ef None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:36.274741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9deb7-f147-40c4-a3e7-c3d1fea790ef None admin] PATH_INFO: `/v3/services/network` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:36.275350 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9deb7-f147-40c4-a3e7-c3d1fea790ef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:36.276784 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9deb7-f147-40c4-a3e7-c3d1fea790ef None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-e9b9deb7-f147-40c4-a3e7-c3d1fea790ef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:36.292413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:36.293471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:36.294422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 267/525] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:36 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:36.304252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9deb7-f147-40c4-a3e7-c3d1fea790ef None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:36.379457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f255ba-96c8-4022-95d7-aa36c0f0bd33 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:36.428926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf5f9b-8c3b-447f-b8c7-5053202292be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:36.429542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf5f9b-8c3b-447f-b8c7-5053202292be None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:36.429817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf5f9b-8c3b-447f-b8c7-5053202292be None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:36.430113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf5f9b-8c3b-447f-b8c7-5053202292be None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:36.430879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf5f9b-8c3b-447f-b8c7-5053202292be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:36.431189 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf5f9b-8c3b-447f-b8c7-5053202292be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:36.432633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf5f9b-8c3b-447f-b8c7-5053202292be None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:36.450411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 259/526] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:36.452317 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf5f9b-8c3b-447f-b8c7-5053202292be None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:36.454702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9deb7-f147-40c4-a3e7-c3d1fea790ef None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:36.494185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8ebd6-16ae-436c-b013-640621766b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:36.494788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8ebd6-16ae-436c-b013-640621766b81 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:36.495061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8ebd6-16ae-436c-b013-640621766b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:36.495330 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8ebd6-16ae-436c-b013-640621766b81 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:36.496061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8ebd6-16ae-436c-b013-640621766b81 None admin] RBAC: Adding query filter params (name=network) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:36.496168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:36.496732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8ebd6-16ae-436c-b013-640621766b81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:36.498144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8ebd6-16ae-436c-b013-640621766b81 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:36.504667 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 268/527] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:12:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:36.521135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8ebd6-16ae-436c-b013-640621766b81 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:36.574399 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:36.580522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf5f9b-8c3b-447f-b8c7-5053202292be None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:36.626418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f0504-1a3f-4825-9833-8a19b86f92a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:36.627190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f0504-1a3f-4825-9833-8a19b86f92a6 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:36.627471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f0504-1a3f-4825-9833-8a19b86f92a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:36.627743 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f0504-1a3f-4825-9833-8a19b86f92a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:36.629384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f0504-1a3f-4825-9833-8a19b86f92a6 None admin] RBAC: Adding query filter params (role.id=695bad5e383d4052a3d6ffa680329cb7, scope.project.id=676e986ff972410d9f498800d313476f, user.id=e476a32b458c4ea0bdc207d0ff4bfd47) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:36.629697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f0504-1a3f-4825-9833-8a19b86f92a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:36.631871 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40f0504-1a3f-4825-9833-8a19b86f92a6 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:36.639568 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8ebd6-16ae-436c-b013-640621766b81 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:36.639743 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 260/528] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:12:36 2022] GET /identity/v3/role_assignments?user.id=e476a32b458c4ea0bdc207d0ff4bfd47&scope.project.id=676e986ff972410d9f498800d313476f&role.id=695bad5e383d4052a3d6ffa680329cb7 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:36.680021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980ff3f-209e-4c33-a69a-52dff86e456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:36.680814 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980ff3f-209e-4c33-a69a-52dff86e456d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:36.681198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980ff3f-209e-4c33-a69a-52dff86e456d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:36.681557 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980ff3f-209e-4c33-a69a-52dff86e456d None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:36.682338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980ff3f-209e-4c33-a69a-52dff86e456d None admin] RBAC: Adding query filter params (type=network) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:36.682734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980ff3f-209e-4c33-a69a-52dff86e456d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:36.684240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980ff3f-209e-4c33-a69a-52dff86e456d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:36.692126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 269/529] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:12:36 2022] GET /identity/v3/services?type=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:36.892376 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121e55d-6e4a-4eca-a509-2b4b6a1ea035 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:36.892582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121e55d-6e4a-4eca-a509-2b4b6a1ea035 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:36.892831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121e55d-6e4a-4eca-a509-2b4b6a1ea035 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:36.894467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 261/530] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:36.900281 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230ca9a-34df-489a-a85b-cb7c5772f693 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:36.900901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230ca9a-34df-489a-a85b-cb7c5772f693 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:36.900901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230ca9a-34df-489a-a85b-cb7c5772f693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:36.923619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-2230ca9a-34df-489a-a85b-cb7c5772f693 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:36.924839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2230ca9a-34df-489a-a85b-cb7c5772f693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:37.006098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 270/531] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:37.352130 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121e55d-6e4a-4eca-a509-2b4b6a1ea035 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:37.479899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:37.484857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121e55d-6e4a-4eca-a509-2b4b6a1ea035 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:37.525842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bb0e2-d957-4a1f-ac5a-6d57ebae888c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:37.526500 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bb0e2-d957-4a1f-ac5a-6d57ebae888c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:37.526773 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bb0e2-d957-4a1f-ac5a-6d57ebae888c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:37.527039 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bb0e2-d957-4a1f-ac5a-6d57ebae888c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:37.543040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bb0e2-d957-4a1f-ac5a-6d57ebae888c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-d21bb0e2-d957-4a1f-ac5a-6d57ebae888c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:37.548932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:37.549992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:37.550868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:37.554736 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 262/532] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:12:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:37.560098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230ca9a-34df-489a-a85b-cb7c5772f693 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:37.639980 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:37.714646 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230ca9a-34df-489a-a85b-cb7c5772f693 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:37.757738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df602b-b74e-4b59-86d9-46b58af3b9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:37.758421 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df602b-b74e-4b59-86d9-46b58af3b9fe None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:37.758696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df602b-b74e-4b59-86d9-46b58af3b9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:37.758962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df602b-b74e-4b59-86d9-46b58af3b9fe None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:37.759613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df602b-b74e-4b59-86d9-46b58af3b9fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:37.759912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df602b-b74e-4b59-86d9-46b58af3b9fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:37.761346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df602b-b74e-4b59-86d9-46b58af3b9fe None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:37.770460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 271/533] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:37.779116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bb0e2-d957-4a1f-ac5a-6d57ebae888c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:37.885527 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bb0e2-d957-4a1f-ac5a-6d57ebae888c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:37.919090 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28873f13-e2cd-436b-8ef9-61048938a77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:37.919633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28873f13-e2cd-436b-8ef9-61048938a77f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:37.919907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28873f13-e2cd-436b-8ef9-61048938a77f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:37.920174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28873f13-e2cd-436b-8ef9-61048938a77f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:37.930843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28873f13-e2cd-436b-8ef9-61048938a77f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:37.932320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28873f13-e2cd-436b-8ef9-61048938a77f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-28873f13-e2cd-436b-8ef9-61048938a77f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:37.946938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:37.947872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:37.949410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:37.949410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:37.949410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:37.949410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 263/534] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:37.955595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df602b-b74e-4b59-86d9-46b58af3b9fe None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:38.111057 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:38.114793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df602b-b74e-4b59-86d9-46b58af3b9fe None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:38.162755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1e7bd-a413-4ec1-a92e-ebe521d6801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:38.163355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e7bd-a413-4ec1-a92e-ebe521d6801a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:38.163631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e7bd-a413-4ec1-a92e-ebe521d6801a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:38.163896 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e7bd-a413-4ec1-a92e-ebe521d6801a None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:38.164523 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1e7bd-a413-4ec1-a92e-ebe521d6801a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:38.164818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1e7bd-a413-4ec1-a92e-ebe521d6801a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:38.166190 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1e7bd-a413-4ec1-a92e-ebe521d6801a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:38.177746 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 272/535] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:38.183048 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28873f13-e2cd-436b-8ef9-61048938a77f None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:38.201457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:38.310737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28873f13-e2cd-436b-8ef9-61048938a77f None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:38.364807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c3246-a248-4816-9f71-a57a5878c087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:38.365431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c3246-a248-4816-9f71-a57a5878c087 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:38.365710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c3246-a248-4816-9f71-a57a5878c087 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:38.365979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c3246-a248-4816-9f71-a57a5878c087 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:38.379430 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c3246-a248-4816-9f71-a57a5878c087 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:38.380873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c3246-a248-4816-9f71-a57a5878c087 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-937c3246-a248-4816-9f71-a57a5878c087 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:38.393613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:38.394637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:38.396237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:38.396237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:38.396237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:38.396237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:38.396237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:38.396237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:38.396237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:38.396237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:38.396237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:12:38.396237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:38.396237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 264/536] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 19 02:12:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:38.400559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1e7bd-a413-4ec1-a92e-ebe521d6801a None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:38.514748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1319f4f-7b0e-49bb-8d78-f7b2f70e92fc None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:38.515106 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1319f4f-7b0e-49bb-8d78-f7b2f70e92fc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:38.515301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1319f4f-7b0e-49bb-8d78-f7b2f70e92fc None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:38.516782 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 265/537] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:38.521726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3edf9-1da4-4342-a63d-0fd8924b9628 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:38.522004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3edf9-1da4-4342-a63d-0fd8924b9628 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:38.522301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3edf9-1da4-4342-a63d-0fd8924b9628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:38.542331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7cc3edf9-1da4-4342-a63d-0fd8924b9628 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:38.544235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7cc3edf9-1da4-4342-a63d-0fd8924b9628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:38.557823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1e7bd-a413-4ec1-a92e-ebe521d6801a None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:38.619899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 266/538] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:38.620451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:38.622730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74520aad-aeef-499a-951c-acf872dd43be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:38.623342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74520aad-aeef-499a-951c-acf872dd43be None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:38.623663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74520aad-aeef-499a-951c-acf872dd43be None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:38.623946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74520aad-aeef-499a-951c-acf872dd43be None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:38.624681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74520aad-aeef-499a-951c-acf872dd43be None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:38.624981 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74520aad-aeef-499a-951c-acf872dd43be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:38.630859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74520aad-aeef-499a-951c-acf872dd43be None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:38.649233 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 273/539] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 19 02:12:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:38.654411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3edf9-1da4-4342-a63d-0fd8924b9628 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:38.768040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3edf9-1da4-4342-a63d-0fd8924b9628 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:38.812163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a1398-f385-4b7e-b8e6-6c4213ef8753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:38.812784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a1398-f385-4b7e-b8e6-6c4213ef8753 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:38.813066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a1398-f385-4b7e-b8e6-6c4213ef8753 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:38.813337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a1398-f385-4b7e-b8e6-6c4213ef8753 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:38.892258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:38.896081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a1398-f385-4b7e-b8e6-6c4213ef8753 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:38.897577 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a1398-f385-4b7e-b8e6-6c4213ef8753 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:38.904494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74520aad-aeef-499a-951c-acf872dd43be None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:38.947202 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:39.009710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 267/540] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 19 02:12:39.019096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a1398-f385-4b7e-b8e6-6c4213ef8753 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:39.101535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74520aad-aeef-499a-951c-acf872dd43be None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:39.173602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d780f-e59a-41c4-8c61-e47fd96fd74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:39.174898 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d780f-e59a-41c4-8c61-e47fd96fd74d None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:39.175381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d780f-e59a-41c4-8c61-e47fd96fd74d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:39.176027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d780f-e59a-41c4-8c61-e47fd96fd74d None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:39.177328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d780f-e59a-41c4-8c61-e47fd96fd74d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:39.179522 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898d780f-e59a-41c4-8c61-e47fd96fd74d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:39.182481 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:39.185559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a1398-f385-4b7e-b8e6-6c4213ef8753 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:39.220155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1814c4-f1d6-4009-8816-b5265fae6d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:39.220781 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1814c4-f1d6-4009-8816-b5265fae6d41 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:39.221054 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1814c4-f1d6-4009-8816-b5265fae6d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:39.221318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1814c4-f1d6-4009-8816-b5265fae6d41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:39.232063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1814c4-f1d6-4009-8816-b5265fae6d41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:39.233528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1814c4-f1d6-4009-8816-b5265fae6d41 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-4c1814c4-f1d6-4009-8816-b5265fae6d41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:39.249283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:39.250214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:39.251123 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:39.251123 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:39.251123 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:39.251123 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 268/541] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:39.251846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff3940-7451-4f02-a258-7c519e83b35c None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:39.252134 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff3940-7451-4f02-a258-7c519e83b35c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:39.252388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff3940-7451-4f02-a258-7c519e83b35c None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:39.253732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 269/542] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:39.258930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f57a7-ef93-4606-bfec-fb9f6d42eb23 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:39.259219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f57a7-ef93-4606-bfec-fb9f6d42eb23 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:39.259479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f57a7-ef93-4606-bfec-fb9f6d42eb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:39.278128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 274/543] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:12:38 2022] POST /identity/v3/services => generated 251 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:39.278128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d780f-e59a-41c4-8c61-e47fd96fd74d None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:39.297811 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-e31f57a7-ef93-4606-bfec-fb9f6d42eb23 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:39.299055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e31f57a7-ef93-4606-bfec-fb9f6d42eb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:39.439769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 270/544] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:39.443655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:39.484897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898d780f-e59a-41c4-8c61-e47fd96fd74d None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:39.535714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b35f3-af10-46e9-9eb4-a2df91bb526a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:39.536249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b35f3-af10-46e9-9eb4-a2df91bb526a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:39.536528 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b35f3-af10-46e9-9eb4-a2df91bb526a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:39.536799 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b35f3-af10-46e9-9eb4-a2df91bb526a None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:39.537450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b35f3-af10-46e9-9eb4-a2df91bb526a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:39.537746 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b35f3-af10-46e9-9eb4-a2df91bb526a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:39.539142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b35f3-af10-46e9-9eb4-a2df91bb526a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:39.553838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 275/545] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:39.559279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f57a7-ef93-4606-bfec-fb9f6d42eb23 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:39.573581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f8a94-e15d-4274-8874-7027ec286d03 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:39.574235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f8a94-e15d-4274-8874-7027ec286d03 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:39.574728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f8a94-e15d-4274-8874-7027ec286d03 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:39.576574 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 276/546] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:12:39.581834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b55b-e262-41d7-a9c6-2eaafdcfc8a3 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:39.582420 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b55b-e262-41d7-a9c6-2eaafdcfc8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:39.582946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e4b55b-e262-41d7-a9c6-2eaafdcfc8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:39.602309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-47e4b55b-e262-41d7-a9c6-2eaafdcfc8a3 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:39.603874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-47e4b55b-e262-41d7-a9c6-2eaafdcfc8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:39.672823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:39.690200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 277/547] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 19 02:12:39.691787 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb53e92-e8ab-4027-93f5-720904067ab5 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:39.692285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb53e92-e8ab-4027-93f5-720904067ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:39.692745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb53e92-e8ab-4027-93f5-720904067ab5 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:39.695033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 278/548] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:39.699927 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c046dc-615e-4797-bc75-f9d02f521434 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:39.700579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c046dc-615e-4797-bc75-f9d02f521434 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:39.702349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c046dc-615e-4797-bc75-f9d02f521434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:39.705826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f57a7-ef93-4606-bfec-fb9f6d42eb23 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:39.724101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-15c046dc-615e-4797-bc75-f9d02f521434 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:39.725286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-15c046dc-615e-4797-bc75-f9d02f521434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:39.745831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4abe68-9fff-426c-9716-17e5e0ce1d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:39.747340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4abe68-9fff-426c-9716-17e5e0ce1d6d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:39.748078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4abe68-9fff-426c-9716-17e5e0ce1d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:39.748572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4abe68-9fff-426c-9716-17e5e0ce1d6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:39.758758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4abe68-9fff-426c-9716-17e5e0ce1d6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:39.760824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4abe68-9fff-426c-9716-17e5e0ce1d6d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-fa4abe68-9fff-426c-9716-17e5e0ce1d6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:39.776614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:39.777523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:39.778415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:39.778415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:39.778415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:39.778415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:39.778415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:39.778415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:39.778415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:39.778415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:39.781201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 271/549] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:39.787003 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4abe68-9fff-426c-9716-17e5e0ce1d6d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:39.796393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:39.802220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 279/550] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 19 02:12:39.803044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c046dc-615e-4797-bc75-f9d02f521434 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:39.956045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c046dc-615e-4797-bc75-f9d02f521434 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:39.993722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4abe68-9fff-426c-9716-17e5e0ce1d6d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:40.002644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90fef8-fa54-4e5a-be2f-516daf6ed3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:40.003294 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90fef8-fa54-4e5a-be2f-516daf6ed3cb None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:40.003570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90fef8-fa54-4e5a-be2f-516daf6ed3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:40.003841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90fef8-fa54-4e5a-be2f-516daf6ed3cb None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:40.004738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90fef8-fa54-4e5a-be2f-516daf6ed3cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:40.005047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90fef8-fa54-4e5a-be2f-516daf6ed3cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:40.006706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90fef8-fa54-4e5a-be2f-516daf6ed3cb None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:40.019739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 280/551] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:40.020627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90fef8-fa54-4e5a-be2f-516daf6ed3cb None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:40.038381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2c35-eb2f-4438-b3a3-2c85806530e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:40.038942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2c35-eb2f-4438-b3a3-2c85806530e9 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:40.039273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2c35-eb2f-4438-b3a3-2c85806530e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:40.039543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52c2c35-eb2f-4438-b3a3-2c85806530e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:40.047175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:40.052152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52c2c35-eb2f-4438-b3a3-2c85806530e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-a52c2c35-eb2f-4438-b3a3-2c85806530e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:40.057471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:40.076298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 272/552] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:40.077676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2c35-eb2f-4438-b3a3-2c85806530e9 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:40.096287 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:40.146513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90fef8-fa54-4e5a-be2f-516daf6ed3cb None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:40.170836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52c2c35-eb2f-4438-b3a3-2c85806530e9 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:40.180660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723639e-5811-4ce2-b8b0-a5e6958d7582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:40.181265 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723639e-5811-4ce2-b8b0-a5e6958d7582 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:40.181543 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723639e-5811-4ce2-b8b0-a5e6958d7582 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:40.181815 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723639e-5811-4ce2-b8b0-a5e6958d7582 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:40.182496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723639e-5811-4ce2-b8b0-a5e6958d7582 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:40.183865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723639e-5811-4ce2-b8b0-a5e6958d7582 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:40.208609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee226a1d-9468-461d-8a7b-606115f8bb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:40.209217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee226a1d-9468-461d-8a7b-606115f8bb2a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:40.209494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee226a1d-9468-461d-8a7b-606115f8bb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:40.209764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee226a1d-9468-461d-8a7b-606115f8bb2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:40.210519 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee226a1d-9468-461d-8a7b-606115f8bb2a None admin] RBAC: Adding query filter params (role.id=d5070bb151d343ca8de32abe373f4a73, scope.project.id=676e986ff972410d9f498800d313476f, user.id=0abf52399a344157ac4e0f9a8212164e) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:40.210826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee226a1d-9468-461d-8a7b-606115f8bb2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:40.212396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee226a1d-9468-461d-8a7b-606115f8bb2a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:40.218786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 273/553] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:12:40 2022] GET /identity/v3/role_assignments?user.id=0abf52399a344157ac4e0f9a8212164e&scope.project.id=676e986ff972410d9f498800d313476f&role.id=d5070bb151d343ca8de32abe373f4a73 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:40.219558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee226a1d-9468-461d-8a7b-606115f8bb2a None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:40.267196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 281/554] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:12:40 2022] POST /identity/v3/services => generated 240 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:40.268393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723639e-5811-4ce2-b8b0-a5e6958d7582 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:40.325912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:40.420893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee226a1d-9468-461d-8a7b-606115f8bb2a None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:40.473849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723639e-5811-4ce2-b8b0-a5e6958d7582 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:40.486380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c44bb8-597b-4231-9c06-c4c939980fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:40.487671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c44bb8-597b-4231-9c06-c4c939980fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:40.488152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c44bb8-597b-4231-9c06-c4c939980fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:40.488642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c44bb8-597b-4231-9c06-c4c939980fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:40.489502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c44bb8-597b-4231-9c06-c4c939980fe6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:40.490001 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c44bb8-597b-4231-9c06-c4c939980fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:40.491677 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c44bb8-597b-4231-9c06-c4c939980fe6 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:40.494280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:40.504625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 274/555] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:40.509426 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c44bb8-597b-4231-9c06-c4c939980fe6 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:40.525786 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a26e4e-9951-4676-a4d1-591fb32eacf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:40.526558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a26e4e-9951-4676-a4d1-591fb32eacf4 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:40.526797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a26e4e-9951-4676-a4d1-591fb32eacf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:40.527035 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a26e4e-9951-4676-a4d1-591fb32eacf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:40.541944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a26e4e-9951-4676-a4d1-591fb32eacf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-a5a26e4e-9951-4676-a4d1-591fb32eacf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:40.548713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:40.550101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:40.551448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 282/556] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:12:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:40.558854 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a26e4e-9951-4676-a4d1-591fb32eacf4 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:40.625676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c44bb8-597b-4231-9c06-c4c939980fe6 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:40.650843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:40.661428 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f1524-1df2-4da3-b287-050355cce603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:40.662088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f1524-1df2-4da3-b287-050355cce603 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:40.662374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f1524-1df2-4da3-b287-050355cce603 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:40.662645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f1524-1df2-4da3-b287-050355cce603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:40.673631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a26e4e-9951-4676-a4d1-591fb32eacf4 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:40.675273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f1524-1df2-4da3-b287-050355cce603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:40.676932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f1524-1df2-4da3-b287-050355cce603 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-fe8f1524-1df2-4da3-b287-050355cce603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:40.692572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:40.693582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:40.694526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:40.694526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:40.694526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:40.704740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 275/557] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:40.704740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f1524-1df2-4da3-b287-050355cce603 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:40.763378 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d795d6-c98f-4ae9-8691-7ab1602ac788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:40.763994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d795d6-c98f-4ae9-8691-7ab1602ac788 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:40.764781 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d795d6-c98f-4ae9-8691-7ab1602ac788 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:40.765058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d795d6-c98f-4ae9-8691-7ab1602ac788 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:40.765745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d795d6-c98f-4ae9-8691-7ab1602ac788 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:40.766025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d795d6-c98f-4ae9-8691-7ab1602ac788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:40.767433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d795d6-c98f-4ae9-8691-7ab1602ac788 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:40.789505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 283/558] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:40.790431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d795d6-c98f-4ae9-8691-7ab1602ac788 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:40.808949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f1524-1df2-4da3-b287-050355cce603 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:40.843282 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07183b-b23a-42fc-92db-f9fe99e2fe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:40.843899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07183b-b23a-42fc-92db-f9fe99e2fe44 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:40.844180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07183b-b23a-42fc-92db-f9fe99e2fe44 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:40.844455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa07183b-b23a-42fc-92db-f9fe99e2fe44 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:40.845110 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07183b-b23a-42fc-92db-f9fe99e2fe44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:40.845411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07183b-b23a-42fc-92db-f9fe99e2fe44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:40.846974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa07183b-b23a-42fc-92db-f9fe99e2fe44 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:40.864045 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 276/559] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:40.865282 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07183b-b23a-42fc-92db-f9fe99e2fe44 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:40.866989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:40.905916 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:40.981070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa07183b-b23a-42fc-92db-f9fe99e2fe44 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:41.017576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d795d6-c98f-4ae9-8691-7ab1602ac788 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:41.040738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d0de4-6abd-4a84-92d1-f77abc6a4742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:41.041367 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d0de4-6abd-4a84-92d1-f77abc6a4742 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:41.041642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d0de4-6abd-4a84-92d1-f77abc6a4742 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:41.041907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d0de4-6abd-4a84-92d1-f77abc6a4742 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:41.051083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d0de4-6abd-4a84-92d1-f77abc6a4742 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:41.052805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d0de4-6abd-4a84-92d1-f77abc6a4742 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-1f2d0de4-6abd-4a84-92d1-f77abc6a4742 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:41.072631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:41.073556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:41.074593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:41.074593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:41.074593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:41.074593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:41.074593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:41.074593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:41.074593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:41.074593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:41.074593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 02:12:41.074593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:41.074593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 277/560] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 19 02:12:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:41.080171 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d0de4-6abd-4a84-92d1-f77abc6a4742 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:41.083100 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d9d1e-5d4b-4aba-91f4-a2eeff9a44a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:41.083765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9d1e-5d4b-4aba-91f4-a2eeff9a44a1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:41.084043 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9d1e-5d4b-4aba-91f4-a2eeff9a44a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:41.084310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9d1e-5d4b-4aba-91f4-a2eeff9a44a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:41.106570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d9d1e-5d4b-4aba-91f4-a2eeff9a44a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:41.108149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d9d1e-5d4b-4aba-91f4-a2eeff9a44a1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-956d9d1e-5d4b-4aba-91f4-a2eeff9a44a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:41.139743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:41.140724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:41.141611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:41.141611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:41.141611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:41.144000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 284/561] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:41.146360 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d9d1e-5d4b-4aba-91f4-a2eeff9a44a1 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:41.252721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:41.263868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d0de4-6abd-4a84-92d1-f77abc6a4742 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:41.306705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:41.324900 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d9d1e-5d4b-4aba-91f4-a2eeff9a44a1 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:41.328593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccf499-d19e-4fc4-bf45-1fd2494a4a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:41.329199 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccf499-d19e-4fc4-bf45-1fd2494a4a22 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:41.329471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccf499-d19e-4fc4-bf45-1fd2494a4a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:41.329737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccf499-d19e-4fc4-bf45-1fd2494a4a22 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:41.330490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccf499-d19e-4fc4-bf45-1fd2494a4a22 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:41.330789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccf499-d19e-4fc4-bf45-1fd2494a4a22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:41.332198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccf499-d19e-4fc4-bf45-1fd2494a4a22 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:41.361522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 278/562] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Oct 19 02:12:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:41.367229 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccf499-d19e-4fc4-bf45-1fd2494a4a22 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:41.392529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4df7-9c96-48c6-b214-35aa72fc4a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:41.393139 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4df7-9c96-48c6-b214-35aa72fc4a79 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:41.393413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4df7-9c96-48c6-b214-35aa72fc4a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:41.393681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf4df7-9c96-48c6-b214-35aa72fc4a79 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:41.394348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf4df7-9c96-48c6-b214-35aa72fc4a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:41.394651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf4df7-9c96-48c6-b214-35aa72fc4a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:41.396011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf4df7-9c96-48c6-b214-35aa72fc4a79 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:41.419642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 285/563] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:41.424055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4df7-9c96-48c6-b214-35aa72fc4a79 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:41.501908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccf499-d19e-4fc4-bf45-1fd2494a4a22 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:41.538624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46d59f-8b69-47ca-906a-97890ed44623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:41.539802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46d59f-8b69-47ca-906a-97890ed44623 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:41.540302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46d59f-8b69-47ca-906a-97890ed44623 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:41.540770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46d59f-8b69-47ca-906a-97890ed44623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:41.552211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46d59f-8b69-47ca-906a-97890ed44623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:41.554454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46d59f-8b69-47ca-906a-97890ed44623 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-cd46d59f-8b69-47ca-906a-97890ed44623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:41.572729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:41.573692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:41.574610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:41.574610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:41.574610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:41.577525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 279/564] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:41.583440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46d59f-8b69-47ca-906a-97890ed44623 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:41.594198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:41.668298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf4df7-9c96-48c6-b214-35aa72fc4a79 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:41.697256 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46d59f-8b69-47ca-906a-97890ed44623 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:41.737203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06759ae6-ed42-430e-b756-3a62e23de8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:41.738300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06759ae6-ed42-430e-b756-3a62e23de8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:41.738832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06759ae6-ed42-430e-b756-3a62e23de8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:41.739338 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06759ae6-ed42-430e-b756-3a62e23de8b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:41.740571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06759ae6-ed42-430e-b756-3a62e23de8b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:41.741142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06759ae6-ed42-430e-b756-3a62e23de8b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:41.742799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06759ae6-ed42-430e-b756-3a62e23de8b7 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:41.759023 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 280/565] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:41.765626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06759ae6-ed42-430e-b756-3a62e23de8b7 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:41.771481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bebd8b-6bcb-4474-b138-b7a7085e6d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:41.772453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bebd8b-6bcb-4474-b138-b7a7085e6d6f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:41.773943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bebd8b-6bcb-4474-b138-b7a7085e6d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:41.774570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bebd8b-6bcb-4474-b138-b7a7085e6d6f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:41.796327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bebd8b-6bcb-4474-b138-b7a7085e6d6f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:41.800606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bebd8b-6bcb-4474-b138-b7a7085e6d6f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-50bebd8b-6bcb-4474-b138-b7a7085e6d6f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:41.823252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:41.824268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:41.825217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:41.825217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:41.825217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:41.825217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:41.825217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:41.825217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:41.825217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:41.825217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:41.825217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 19 02:12:41.825217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:41.825217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 286/566] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:12:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:41.829857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bebd8b-6bcb-4474-b138-b7a7085e6d6f None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:41.850318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:41.868930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:41.909798 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06759ae6-ed42-430e-b756-3a62e23de8b7 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:41.948758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a837c-5e5b-429f-8833-4dfb1e5c332c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:41.949737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a837c-5e5b-429f-8833-4dfb1e5c332c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:41.950285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a837c-5e5b-429f-8833-4dfb1e5c332c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:41.950933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a837c-5e5b-429f-8833-4dfb1e5c332c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:41.959661 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a837c-5e5b-429f-8833-4dfb1e5c332c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:41.960806 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bebd8b-6bcb-4474-b138-b7a7085e6d6f None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:41.961624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a837c-5e5b-429f-8833-4dfb1e5c332c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-634a837c-5e5b-429f-8833-4dfb1e5c332c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:41.974720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:41.975625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:41.976640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:41.976640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:41.976640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:41.976640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:41.976640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:41.976640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:41.976640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:41.976640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:41.979261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 281/567] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:41.980864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394ea3-7a26-4397-947d-9e57f11928bc None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:41.981333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394ea3-7a26-4397-947d-9e57f11928bc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:41.981807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f394ea3-7a26-4397-947d-9e57f11928bc None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:41.987529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 282/568] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:41.988996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394ea3-7a26-4397-947d-9e57f11928bc None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:42.001585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc23b79-29f0-47c4-9f6f-ade3faa56ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:42.002747 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc23b79-29f0-47c4-9f6f-ade3faa56ecd None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:42.003664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc23b79-29f0-47c4-9f6f-ade3faa56ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:42.003664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc23b79-29f0-47c4-9f6f-ade3faa56ecd None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:42.004399 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc23b79-29f0-47c4-9f6f-ade3faa56ecd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:42.004717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc23b79-29f0-47c4-9f6f-ade3faa56ecd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:42.006320 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc23b79-29f0-47c4-9f6f-ade3faa56ecd None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:42.034093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 287/569] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 19 02:12:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:42.034093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25565a7e-7d8e-4e36-8882-4b56912596ed None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:42.034093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25565a7e-7d8e-4e36-8882-4b56912596ed None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:42.034093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25565a7e-7d8e-4e36-8882-4b56912596ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:42.054325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-25565a7e-7d8e-4e36-8882-4b56912596ed None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:42.055774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-25565a7e-7d8e-4e36-8882-4b56912596ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:42.116278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f394ea3-7a26-4397-947d-9e57f11928bc None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:42.123206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:42.138315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 288/570] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:42.139943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25565a7e-7d8e-4e36-8882-4b56912596ed None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:42.146307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:42.154198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5b139-b5be-49da-b4de-d19caff07e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:42.155594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5b139-b5be-49da-b4de-d19caff07e3c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:42.156037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5b139-b5be-49da-b4de-d19caff07e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:42.156449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5b139-b5be-49da-b4de-d19caff07e3c None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:42.157281 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5b139-b5be-49da-b4de-d19caff07e3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:42.157730 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5b139-b5be-49da-b4de-d19caff07e3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:42.159308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5b139-b5be-49da-b4de-d19caff07e3c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:42.173012 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 283/571] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:42.178774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5b139-b5be-49da-b4de-d19caff07e3c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:42.283809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5b139-b5be-49da-b4de-d19caff07e3c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:42.319665 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f07c1-622b-448a-9ac2-1f4ee4a84b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:42.320612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f07c1-622b-448a-9ac2-1f4ee4a84b08 None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:42.321051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f07c1-622b-448a-9ac2-1f4ee4a84b08 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:42.321462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f07c1-622b-448a-9ac2-1f4ee4a84b08 None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/717c74cb769148a5b465079add961892/roles/695bad5e383d4052a3d6ffa680329cb7` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:42.339898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f07c1-622b-448a-9ac2-1f4ee4a84b08 None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=717c74cb769148a5b465079add961892, role_id=695bad5e383d4052a3d6ffa680329cb7)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:42.341524 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f07c1-622b-448a-9ac2-1f4ee4a84b08 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:42.380316 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25565a7e-7d8e-4e36-8882-4b56912596ed None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:42.416905 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe43604-8684-4981-b723-d40d0a0eb787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:42.417777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe43604-8684-4981-b723-d40d0a0eb787 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:42.418251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe43604-8684-4981-b723-d40d0a0eb787 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:42.418661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe43604-8684-4981-b723-d40d0a0eb787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:42.427381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 284/572] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:12:42 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/717c74cb769148a5b465079add961892/roles/695bad5e383d4052a3d6ffa680329cb7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:42.437535 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f07c1-622b-448a-9ac2-1f4ee4a84b08 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:42.438372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe43604-8684-4981-b723-d40d0a0eb787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:42.440359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe43604-8684-4981-b723-d40d0a0eb787 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-8fe43604-8684-4981-b723-d40d0a0eb787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:42.465537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:42.470007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:42.471141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:42.471141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:42.471141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:42.471141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 289/573] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:42.473062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe43604-8684-4981-b723-d40d0a0eb787 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:42.500275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:42.547431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:42.603532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe43604-8684-4981-b723-d40d0a0eb787 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:42.652134 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f07c1-622b-448a-9ac2-1f4ee4a84b08 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:42.667108 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f374b3-265d-4e80-a4d1-fa9b163b73f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:42.667756 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f374b3-265d-4e80-a4d1-fa9b163b73f4 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:42.668035 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f374b3-265d-4e80-a4d1-fa9b163b73f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:42.668304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f374b3-265d-4e80-a4d1-fa9b163b73f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:42.669053 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f374b3-265d-4e80-a4d1-fa9b163b73f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:42.669453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f374b3-265d-4e80-a4d1-fa9b163b73f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:42.670991 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f374b3-265d-4e80-a4d1-fa9b163b73f4 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:42.684050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 290/574] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:42.691307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9b31-391d-4181-9534-de9edce46e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:42.691801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f374b3-265d-4e80-a4d1-fa9b163b73f4 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:42.691953 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9b31-391d-4181-9534-de9edce46e79 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:42.692654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9b31-391d-4181-9534-de9edce46e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:42.692654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9b31-391d-4181-9534-de9edce46e79 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:42.694721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc9b31-391d-4181-9534-de9edce46e79 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:42.694721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc9b31-391d-4181-9534-de9edce46e79 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-d7fc9b31-391d-4181-9534-de9edce46e79 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:42.705516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:42.706543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:42.707417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 285/575] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Oct 19 02:12:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:42.717294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9b31-391d-4181-9534-de9edce46e79 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:42.826108 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f374b3-265d-4e80-a4d1-fa9b163b73f4 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:42.841114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:42.844453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:42.845043 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9b31-391d-4181-9534-de9edce46e79 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:42.884403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8abae2-1e2c-4669-ae5a-4a0b8380f208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:42.885088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8abae2-1e2c-4669-ae5a-4a0b8380f208 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:42.885365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8abae2-1e2c-4669-ae5a-4a0b8380f208 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:42.885636 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8abae2-1e2c-4669-ae5a-4a0b8380f208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:42.886618 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9a1db-82d1-45f2-aad7-14bdd28a1912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:42.887340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9a1db-82d1-45f2-aad7-14bdd28a1912 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:42.893098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9a1db-82d1-45f2-aad7-14bdd28a1912 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:42.893098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9a1db-82d1-45f2-aad7-14bdd28a1912 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:42.893098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9a1db-82d1-45f2-aad7-14bdd28a1912 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:42.893098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9a1db-82d1-45f2-aad7-14bdd28a1912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:42.893098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9a1db-82d1-45f2-aad7-14bdd28a1912 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:42.895730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8abae2-1e2c-4669-ae5a-4a0b8380f208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:42.897242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8abae2-1e2c-4669-ae5a-4a0b8380f208 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:42.897629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 286/576] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 19 02:12:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:42.899194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffa822-01d9-4a1f-8909-9af3ea17e947 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:42.904290 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffa822-01d9-4a1f-8909-9af3ea17e947 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:42.904617 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffa822-01d9-4a1f-8909-9af3ea17e947 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:42.906653 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 287/577] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:42.907653 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffa822-01d9-4a1f-8909-9af3ea17e947 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-4b8abae2-1e2c-4669-ae5a-4a0b8380f208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:42.916842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:42.918182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:42.919846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:42.919846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:42.919846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:42.919846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:42.919846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:42.919846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:42.919846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:42.919846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:42.919846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 291/578] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:42.921589 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490aa1-598b-42d6-9e8c-7e384ccd0335 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:42.921740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490aa1-598b-42d6-9e8c-7e384ccd0335 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:42.926110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0490aa1-598b-42d6-9e8c-7e384ccd0335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:42.944460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b0490aa1-598b-42d6-9e8c-7e384ccd0335 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:42.945811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b0490aa1-598b-42d6-9e8c-7e384ccd0335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:43.025814 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 292/579] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:43.026693 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0490aa1-598b-42d6-9e8c-7e384ccd0335 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:43.035802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffa822-01d9-4a1f-8909-9af3ea17e947 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:43.072639 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a70f3-1a88-44ae-b23d-edc6d7a8f0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:43.073252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a70f3-1a88-44ae-b23d-edc6d7a8f0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:43.073532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a70f3-1a88-44ae-b23d-edc6d7a8f0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:43.073802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a70f3-1a88-44ae-b23d-edc6d7a8f0d2 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:43.074608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a70f3-1a88-44ae-b23d-edc6d7a8f0d2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:43.074914 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a70f3-1a88-44ae-b23d-edc6d7a8f0d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:43.076441 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a70f3-1a88-44ae-b23d-edc6d7a8f0d2 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:43.082424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 288/580] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 19 02:12:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:43.083879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190cea-8802-4981-b07b-a6d8d0b38c69 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:43.084168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190cea-8802-4981-b07b-a6d8d0b38c69 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:43.084442 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5190cea-8802-4981-b07b-a6d8d0b38c69 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:43.086081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 289/581] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:43.087977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5190cea-8802-4981-b07b-a6d8d0b38c69 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:43.130690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:43.147950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:43.151271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0490aa1-598b-42d6-9e8c-7e384ccd0335 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:43.197343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e153fe2-d846-4dd8-ab67-795e1ef92c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:43.198365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e153fe2-d846-4dd8-ab67-795e1ef92c9b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:43.199291 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e153fe2-d846-4dd8-ab67-795e1ef92c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:43.200037 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e153fe2-d846-4dd8-ab67-795e1ef92c9b None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:43.200926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e153fe2-d846-4dd8-ab67-795e1ef92c9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:43.201624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e153fe2-d846-4dd8-ab67-795e1ef92c9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:43.204112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e153fe2-d846-4dd8-ab67-795e1ef92c9b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:43.220048 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 293/582] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:43.222228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4bba5-8465-411c-9775-41cdc3fb3bfa None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:43.222512 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4bba5-8465-411c-9775-41cdc3fb3bfa None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:43.222784 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4bba5-8465-411c-9775-41cdc3fb3bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:43.228520 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5190cea-8802-4981-b07b-a6d8d0b38c69 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:43.241258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e1b4bba5-8465-411c-9775-41cdc3fb3bfa None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:43.242523 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e1b4bba5-8465-411c-9775-41cdc3fb3bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:43.264993 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27904c71-e203-42a2-ab5c-7de9d7ca0a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:43.265878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27904c71-e203-42a2-ab5c-7de9d7ca0a92 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:43.266378 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27904c71-e203-42a2-ab5c-7de9d7ca0a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:43.266915 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27904c71-e203-42a2-ab5c-7de9d7ca0a92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:43.267841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27904c71-e203-42a2-ab5c-7de9d7ca0a92 None admin] RBAC: Adding query filter params (interface=public, service_id=b122eb5a6e224da5b4e755f6e000cdeb, region_id=RegionOne) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:43.268346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27904c71-e203-42a2-ab5c-7de9d7ca0a92 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:43.269976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27904c71-e203-42a2-ab5c-7de9d7ca0a92 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:43.278193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 290/583] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 19 02:12:43 2022] GET /identity/v3/endpoints?service_id=b122eb5a6e224da5b4e755f6e000cdeb&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:43.279294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27904c71-e203-42a2-ab5c-7de9d7ca0a92 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:43.318069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 294/584] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:43.318069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4bba5-8465-411c-9775-41cdc3fb3bfa None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:43.396691 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:43.411535 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27904c71-e203-42a2-ab5c-7de9d7ca0a92 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:43.411703 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:43.423177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4bba5-8465-411c-9775-41cdc3fb3bfa None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:43.448588 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e05b85-362d-4fcf-a196-354d12d2e90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:43.449483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e05b85-362d-4fcf-a196-354d12d2e90b None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:43.449927 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e05b85-362d-4fcf-a196-354d12d2e90b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:43.450374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e05b85-362d-4fcf-a196-354d12d2e90b None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/e476a32b458c4ea0bdc207d0ff4bfd47/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:43.453395 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e05b85-362d-4fcf-a196-354d12d2e90b None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=e476a32b458c4ea0bdc207d0ff4bfd47, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:43.455042 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e05b85-362d-4fcf-a196-354d12d2e90b None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:43.460320 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a9e52-7864-4879-bf1c-81df61bb7c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:43.461074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a9e52-7864-4879-bf1c-81df61bb7c43 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:43.461436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a9e52-7864-4879-bf1c-81df61bb7c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:43.461822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a9e52-7864-4879-bf1c-81df61bb7c43 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:43.462504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a9e52-7864-4879-bf1c-81df61bb7c43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:43.464235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a9e52-7864-4879-bf1c-81df61bb7c43 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:43.474325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 295/585] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 19 02:12:43 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:43.475669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a9e52-7864-4879-bf1c-81df61bb7c43 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:43.580418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 291/586] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:12:43 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/e476a32b458c4ea0bdc207d0ff4bfd47/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:43.599992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a9e52-7864-4879-bf1c-81df61bb7c43 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:43.611497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e05b85-362d-4fcf-a196-354d12d2e90b None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:43.641092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c9258-2262-473e-a350-cbfbc320415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:43.642074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c9258-2262-473e-a350-cbfbc320415b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:43.642557 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c9258-2262-473e-a350-cbfbc320415b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:43.642996 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c9258-2262-473e-a350-cbfbc320415b None admin] PATH_INFO: `/v3/services/network` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:43.643784 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c9258-2262-473e-a350-cbfbc320415b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:43.645395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c9258-2262-473e-a350-cbfbc320415b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-476c9258-2262-473e-a350-cbfbc320415b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:43.656685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:43.657808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 02:12:43.658727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:43.661532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 296/587] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:43 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:43.668792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c9258-2262-473e-a350-cbfbc320415b None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:43.702790 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:43.774452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e05b85-362d-4fcf-a196-354d12d2e90b None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:43.814892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c9258-2262-473e-a350-cbfbc320415b None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:43.826276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b6e1f-bd2c-48c8-ad29-28acd11ae216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:43.827695 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b6e1f-bd2c-48c8-ad29-28acd11ae216 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:43.828435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b6e1f-bd2c-48c8-ad29-28acd11ae216 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:43.829164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b6e1f-bd2c-48c8-ad29-28acd11ae216 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:43.829933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b6e1f-bd2c-48c8-ad29-28acd11ae216 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:43.831541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b6e1f-bd2c-48c8-ad29-28acd11ae216 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-fd9b6e1f-bd2c-48c8-ad29-28acd11ae216 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:43.842386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:43.843399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 02:12:43.844257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:43.847508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 292/588] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Oct 19 02:12:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:43.853153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b6e1f-bd2c-48c8-ad29-28acd11ae216 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:43.855736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818296a-d354-4d0e-9c37-b805f39a4583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:43.855970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818296a-d354-4d0e-9c37-b805f39a4583 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:43.857334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818296a-d354-4d0e-9c37-b805f39a4583 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:43.857613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818296a-d354-4d0e-9c37-b805f39a4583 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:43.858793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818296a-d354-4d0e-9c37-b805f39a4583 None admin] RBAC: Adding query filter params (name=network) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:43.859095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818296a-d354-4d0e-9c37-b805f39a4583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:43.860802 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818296a-d354-4d0e-9c37-b805f39a4583 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:43.871168 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 297/589] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:12:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:43.876940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818296a-d354-4d0e-9c37-b805f39a4583 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:43.878779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:43.964648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:43.970733 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b6e1f-bd2c-48c8-ad29-28acd11ae216 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:43.991689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818296a-d354-4d0e-9c37-b805f39a4583 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:44.027930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf0a24-3cc6-4768-8e88-e623a1e88c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:44.028839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf0a24-3cc6-4768-8e88-e623a1e88c00 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:44.029316 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf0a24-3cc6-4768-8e88-e623a1e88c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:44.029774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf0a24-3cc6-4768-8e88-e623a1e88c00 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:44.031127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf0a24-3cc6-4768-8e88-e623a1e88c00 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:44.031603 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf0a24-3cc6-4768-8e88-e623a1e88c00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:44.033617 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf0a24-3cc6-4768-8e88-e623a1e88c00 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:44.041107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 293/590] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 19 02:12:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:44.042797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6207ff-f952-4871-877a-cc2b8f6fd174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:44.043640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6207ff-f952-4871-877a-cc2b8f6fd174 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:44.044094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6207ff-f952-4871-877a-cc2b8f6fd174 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:44.044536 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6207ff-f952-4871-877a-cc2b8f6fd174 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:44.045343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6207ff-f952-4871-877a-cc2b8f6fd174 None admin] RBAC: Adding query filter params (type=network) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:44.045893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6207ff-f952-4871-877a-cc2b8f6fd174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:44.047478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6207ff-f952-4871-877a-cc2b8f6fd174 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:44.054073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf0a24-3cc6-4768-8e88-e623a1e88c00 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:44.065499 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 298/591] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:12:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:44.071421 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6207ff-f952-4871-877a-cc2b8f6fd174 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:44.206456 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:44.226825 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf0a24-3cc6-4768-8e88-e623a1e88c00 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:44.261493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6207ff-f952-4871-877a-cc2b8f6fd174 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:44.280487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4944698-01b6-4745-81fb-c84a3aaf9d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:44.281102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4944698-01b6-4745-81fb-c84a3aaf9d57 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:44.281385 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4944698-01b6-4745-81fb-c84a3aaf9d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:44.281658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4944698-01b6-4745-81fb-c84a3aaf9d57 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:44.282316 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4944698-01b6-4745-81fb-c84a3aaf9d57 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:44.282618 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4944698-01b6-4745-81fb-c84a3aaf9d57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:44.283982 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4944698-01b6-4745-81fb-c84a3aaf9d57 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:44.302483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 294/592] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 19 02:12:44 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:44.328502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c8e36-a9e3-4057-ab58-5c455572805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:44.329120 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c8e36-a9e3-4057-ab58-5c455572805f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:44.329399 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c8e36-a9e3-4057-ab58-5c455572805f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:44.329673 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c8e36-a9e3-4057-ab58-5c455572805f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:44.330402 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c8e36-a9e3-4057-ab58-5c455572805f None admin] RBAC: Adding query filter params (interface=public, service_id=5840421fb290453cb78dc3da00a79933, region_id=RegionOne) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:44.330708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c8e36-a9e3-4057-ab58-5c455572805f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:44.332076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c8e36-a9e3-4057-ab58-5c455572805f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:44.341157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 299/593] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 19 02:12:44 2022] GET /identity/v3/endpoints?service_id=5840421fb290453cb78dc3da00a79933&interface=public®ion_id=RegionOne => generated 194 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:44.351252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4944698-01b6-4745-81fb-c84a3aaf9d57 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:44.482904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4944698-01b6-4745-81fb-c84a3aaf9d57 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:44.525545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6a5d2-4ece-4306-b5e9-85fac0f31287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:44.526285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6a5d2-4ece-4306-b5e9-85fac0f31287 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:44.526629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6a5d2-4ece-4306-b5e9-85fac0f31287 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:44.526970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6a5d2-4ece-4306-b5e9-85fac0f31287 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:44.527708 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6a5d2-4ece-4306-b5e9-85fac0f31287 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:44.529181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6a5d2-4ece-4306-b5e9-85fac0f31287 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:44.534813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 295/594] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 19 02:12:44 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:45.104517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca57753-f9f6-4890-ac8a-dd9bf9d7d726 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:45.105040 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca57753-f9f6-4890-ac8a-dd9bf9d7d726 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:45.109642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca57753-f9f6-4890-ac8a-dd9bf9d7d726 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:45.114196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 300/595] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:45.119416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288ef8f-d45b-4675-b6a5-bc46218f4db4 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:45.119829 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288ef8f-d45b-4675-b6a5-bc46218f4db4 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:45.119996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288ef8f-d45b-4675-b6a5-bc46218f4db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:45.147602 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c288ef8f-d45b-4675-b6a5-bc46218f4db4 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:45.148805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c288ef8f-d45b-4675-b6a5-bc46218f4db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:45.150882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:45.238574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 296/596] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:45.550867 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca57753-f9f6-4890-ac8a-dd9bf9d7d726 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:45.563157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:45.685482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca57753-f9f6-4890-ac8a-dd9bf9d7d726 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:45.731154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f1015-bdf9-4ef8-a0cc-e425f27f197f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:45.731777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f1015-bdf9-4ef8-a0cc-e425f27f197f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:45.732055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f1015-bdf9-4ef8-a0cc-e425f27f197f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:45.732326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f1015-bdf9-4ef8-a0cc-e425f27f197f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:45.749880 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f1015-bdf9-4ef8-a0cc-e425f27f197f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-fe5f1015-bdf9-4ef8-a0cc-e425f27f197f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:45.754728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:45.755705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:45.756585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 301/597] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:12:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:45.761752 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288ef8f-d45b-4675-b6a5-bc46218f4db4 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:45.890403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288ef8f-d45b-4675-b6a5-bc46218f4db4 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:45.933061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93ce0e-9c65-4b13-af02-8ee4cc313e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:45.933729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93ce0e-9c65-4b13-af02-8ee4cc313e11 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:45.934011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93ce0e-9c65-4b13-af02-8ee4cc313e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:45.934307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93ce0e-9c65-4b13-af02-8ee4cc313e11 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:45.934968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93ce0e-9c65-4b13-af02-8ee4cc313e11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:45.935269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93ce0e-9c65-4b13-af02-8ee4cc313e11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:45.936696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab93ce0e-9c65-4b13-af02-8ee4cc313e11 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:45.946368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:45.949738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 297/598] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:12:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:45.956029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f1015-bdf9-4ef8-a0cc-e425f27f197f None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:46.041139 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:46.091399 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f1015-bdf9-4ef8-a0cc-e425f27f197f None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:46.123631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57bd3b-fd72-41a7-a2eb-d32ec9a4995d None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:46.124115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57bd3b-fd72-41a7-a2eb-d32ec9a4995d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:46.127783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57bd3b-fd72-41a7-a2eb-d32ec9a4995d None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:46.129124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 298/599] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:46.135927 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b928b7-26b6-48e6-8a15-d073366a6959 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:46.136210 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b928b7-26b6-48e6-8a15-d073366a6959 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:46.136485 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b928b7-26b6-48e6-8a15-d073366a6959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:46.167947 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-d1b928b7-26b6-48e6-8a15-d073366a6959 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:46.169060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d1b928b7-26b6-48e6-8a15-d073366a6959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:46.172524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fa0e1-fc9e-485f-a105-7fa1f550cc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:46.173158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fa0e1-fc9e-485f-a105-7fa1f550cc6f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:46.173430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fa0e1-fc9e-485f-a105-7fa1f550cc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:46.173697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fa0e1-fc9e-485f-a105-7fa1f550cc6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:46.197647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fa0e1-fc9e-485f-a105-7fa1f550cc6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:46.199133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6fa0e1-fc9e-485f-a105-7fa1f550cc6f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-be6fa0e1-fc9e-485f-a105-7fa1f550cc6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:46.222595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:46.223615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:46.224503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:46.224503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:46.224503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:46.228148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 302/600] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:46.229293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fa0e1-fc9e-485f-a105-7fa1f550cc6f None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:46.253502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 299/601] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:46.408422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fa0e1-fc9e-485f-a105-7fa1f550cc6f None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:46.431296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:46.447647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bda44-494c-4455-9119-871fa002dc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:46.455333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bda44-494c-4455-9119-871fa002dc7d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:46.455333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bda44-494c-4455-9119-871fa002dc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:46.455333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433bda44-494c-4455-9119-871fa002dc7d None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:46.455333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bda44-494c-4455-9119-871fa002dc7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:46.455333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bda44-494c-4455-9119-871fa002dc7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:46.455333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433bda44-494c-4455-9119-871fa002dc7d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:46.469582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 303/602] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:46.475295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b928b7-26b6-48e6-8a15-d073366a6959 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:46.529403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bda44-494c-4455-9119-871fa002dc7d None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:46.635280 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b928b7-26b6-48e6-8a15-d073366a6959 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:46.673426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433bda44-494c-4455-9119-871fa002dc7d None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:46.695762 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:46.703271 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce0197-eb16-4997-a968-939b92ec6734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:46.704115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce0197-eb16-4997-a968-939b92ec6734 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:46.704455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce0197-eb16-4997-a968-939b92ec6734 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:46.704799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce0197-eb16-4997-a968-939b92ec6734 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:46.713593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce0197-eb16-4997-a968-939b92ec6734 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:46.715543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce0197-eb16-4997-a968-939b92ec6734 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:46.719288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d109a1-8afa-4d85-bc35-1c5ba4572e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:46.720965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d109a1-8afa-4d85-bc35-1c5ba4572e7d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:46.720965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d109a1-8afa-4d85-bc35-1c5ba4572e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:46.720965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d109a1-8afa-4d85-bc35-1c5ba4572e7d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:46.721657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d109a1-8afa-4d85-bc35-1c5ba4572e7d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:46.723443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d109a1-8afa-4d85-bc35-1c5ba4572e7d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-f6ce0197-eb16-4997-a968-939b92ec6734 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:46.729878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:46.730853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:46.731792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:46.731792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:46.731792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:46.731792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:46.731792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:46.731792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:46.731792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:46.731792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:46.731792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 02:12:46.731792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:46.731792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 300/603] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 19 02:12:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:46.733808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2565f3c-aac8-4a4f-bf69-04e51c75100c None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:46.734175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2565f3c-aac8-4a4f-bf69-04e51c75100c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:46.734510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2565f3c-aac8-4a4f-bf69-04e51c75100c None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-a2d109a1-8afa-4d85-bc35-1c5ba4572e7d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:46.737225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:46.738246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 19 02:12:46.739077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:46.740271 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 301/604] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:46.740935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2565f3c-aac8-4a4f-bf69-04e51c75100c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:46.742316 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 304/605] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Oct 19 02:12:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:46.743875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeae5ad-0b2d-421d-84f9-dd5de5a8fe15 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:46.744362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeae5ad-0b2d-421d-84f9-dd5de5a8fe15 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:46.744874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeae5ad-0b2d-421d-84f9-dd5de5a8fe15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:46.767849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-adeae5ad-0b2d-421d-84f9-dd5de5a8fe15 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:46.769894 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-adeae5ad-0b2d-421d-84f9-dd5de5a8fe15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:46.831691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:46.869331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2565f3c-aac8-4a4f-bf69-04e51c75100c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:46.870304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 305/606] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:46.871254 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeae5ad-0b2d-421d-84f9-dd5de5a8fe15 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:46.907798 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d3bb5-b3f4-4e57-8341-44a790a15cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:46.909197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d3bb5-b3f4-4e57-8341-44a790a15cbb None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:46.909885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d3bb5-b3f4-4e57-8341-44a790a15cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:46.910417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d3bb5-b3f4-4e57-8341-44a790a15cbb None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:46.911357 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d3bb5-b3f4-4e57-8341-44a790a15cbb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:46.911866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d3bb5-b3f4-4e57-8341-44a790a15cbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:46.913483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d3bb5-b3f4-4e57-8341-44a790a15cbb None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:46.935433 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 302/607] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Oct 19 02:12:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:46.941525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d3bb5-b3f4-4e57-8341-44a790a15cbb None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:47.025351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:47.070405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeae5ad-0b2d-421d-84f9-dd5de5a8fe15 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:47.101796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d3bb5-b3f4-4e57-8341-44a790a15cbb None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:47.114632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce8886-b57b-4d4e-80e3-27d639b00d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:47.115705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce8886-b57b-4d4e-80e3-27d639b00d45 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.115989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce8886-b57b-4d4e-80e3-27d639b00d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.116260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce8886-b57b-4d4e-80e3-27d639b00d45 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.116938 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce8886-b57b-4d4e-80e3-27d639b00d45 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:47.117244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce8886-b57b-4d4e-80e3-27d639b00d45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:47.119022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce8886-b57b-4d4e-80e3-27d639b00d45 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:47.125333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 306/608] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 19 02:12:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:47.130611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9da7b-f330-4f67-90f4-c85f3d6ce982 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.131276 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9da7b-f330-4f67-90f4-c85f3d6ce982 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.131546 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9da7b-f330-4f67-90f4-c85f3d6ce982 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.132909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 307/609] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:47.135964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9da7b-f330-4f67-90f4-c85f3d6ce982 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:47.165910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337e2be-4cb2-4fdd-b026-4fa274581eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:47.167516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337e2be-4cb2-4fdd-b026-4fa274581eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.168097 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337e2be-4cb2-4fdd-b026-4fa274581eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.168600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337e2be-4cb2-4fdd-b026-4fa274581eb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.187858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:47.252324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9da7b-f330-4f67-90f4-c85f3d6ce982 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:47.252549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337e2be-4cb2-4fdd-b026-4fa274581eb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:47.253859 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337e2be-4cb2-4fdd-b026-4fa274581eb6 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:47.287530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15689238-1cc6-45ac-a1a2-d2cf5839ff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:47.288056 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15689238-1cc6-45ac-a1a2-d2cf5839ff9b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.288332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15689238-1cc6-45ac-a1a2-d2cf5839ff9b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.288607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15689238-1cc6-45ac-a1a2-d2cf5839ff9b None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.289231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15689238-1cc6-45ac-a1a2-d2cf5839ff9b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:47.289532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15689238-1cc6-45ac-a1a2-d2cf5839ff9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:47.290918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15689238-1cc6-45ac-a1a2-d2cf5839ff9b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:47.295548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 308/610] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Oct 19 02:12:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:47.297166 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec516a-2731-4dc0-82f9-778ceba1ecf9 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.297445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec516a-2731-4dc0-82f9-778ceba1ecf9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.297713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec516a-2731-4dc0-82f9-778ceba1ecf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.307272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:47.315104 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-dfec516a-2731-4dc0-82f9-778ceba1ecf9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:47.316283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-dfec516a-2731-4dc0-82f9-778ceba1ecf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:47.357255 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 303/611] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 02:12:47.358619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337e2be-4cb2-4fdd-b026-4fa274581eb6 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:47.380936 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 309/612] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:47.382555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01327981-e640-4f08-97c2-e9206c32a287 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.382843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01327981-e640-4f08-97c2-e9206c32a287 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.383113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01327981-e640-4f08-97c2-e9206c32a287 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.384337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 310/613] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:47.387394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01327981-e640-4f08-97c2-e9206c32a287 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:47.425778 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:47.459749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337e2be-4cb2-4fdd-b026-4fa274581eb6 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:47.493304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9c8d5-b5f7-4ba4-aa42-b04b4141231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:47.493954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9c8d5-b5f7-4ba4-aa42-b04b4141231e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.494231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9c8d5-b5f7-4ba4-aa42-b04b4141231e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.494521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb9c8d5-b5f7-4ba4-aa42-b04b4141231e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.500110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01327981-e640-4f08-97c2-e9206c32a287 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:47.504284 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb9c8d5-b5f7-4ba4-aa42-b04b4141231e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-cfb9c8d5-b5f7-4ba4-aa42-b04b4141231e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:47.509089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:47.510247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 304/614] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:12:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:47.511277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9c8d5-b5f7-4ba4-aa42-b04b4141231e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:47.523157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:47.542189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bd535-1e13-43fb-b871-4794fd68132b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:47.542728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bd535-1e13-43fb-b871-4794fd68132b None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.543000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bd535-1e13-43fb-b871-4794fd68132b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.543273 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bd535-1e13-43fb-b871-4794fd68132b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.543930 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bd535-1e13-43fb-b871-4794fd68132b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:47.545494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9bd535-1e13-43fb-b871-4794fd68132b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:47.612333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb9c8d5-b5f7-4ba4-aa42-b04b4141231e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:47.644696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57547ec6-1d06-4c63-b667-4e008345cc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:47.645313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57547ec6-1d06-4c63-b667-4e008345cc59 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.645587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57547ec6-1d06-4c63-b667-4e008345cc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.645856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57547ec6-1d06-4c63-b667-4e008345cc59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.656323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57547ec6-1d06-4c63-b667-4e008345cc59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:47.657753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57547ec6-1d06-4c63-b667-4e008345cc59 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:47.658262 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 311/615] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:12:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:47.659710 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aed923-5292-404e-bcb1-90ee9a837310 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.660124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aed923-5292-404e-bcb1-90ee9a837310 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.660482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aed923-5292-404e-bcb1-90ee9a837310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-57547ec6-1d06-4c63-b667-4e008345cc59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:47.674984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:47.680912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:47.681846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:47.681846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:47.681846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:47.681846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 305/616] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:47.681846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57547ec6-1d06-4c63-b667-4e008345cc59 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:47.689156 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-73aed923-5292-404e-bcb1-90ee9a837310 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:47.691032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-73aed923-5292-404e-bcb1-90ee9a837310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:47.745337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:47.777626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 312/617] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:47.778657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aed923-5292-404e-bcb1-90ee9a837310 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:47.794570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57547ec6-1d06-4c63-b667-4e008345cc59 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:47.806364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:47.850320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f1c03-e74a-4331-a3fd-7668ceaf668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:47.851064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f1c03-e74a-4331-a3fd-7668ceaf668f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.851462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f1c03-e74a-4331-a3fd-7668ceaf668f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.851739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f1c03-e74a-4331-a3fd-7668ceaf668f None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.852498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f1c03-e74a-4331-a3fd-7668ceaf668f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:47.852803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f1c03-e74a-4331-a3fd-7668ceaf668f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:47.854197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f1c03-e74a-4331-a3fd-7668ceaf668f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:47.863701 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 306/618] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:12:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:47.864619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f1c03-e74a-4331-a3fd-7668ceaf668f None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:47.910917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aed923-5292-404e-bcb1-90ee9a837310 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:47.958745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82542309-8545-4082-9dbf-8891808f4304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:47.959519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82542309-8545-4082-9dbf-8891808f4304 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:47.959888 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82542309-8545-4082-9dbf-8891808f4304 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:47.960909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82542309-8545-4082-9dbf-8891808f4304 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:47.961654 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82542309-8545-4082-9dbf-8891808f4304 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:47.963352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82542309-8545-4082-9dbf-8891808f4304 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:47.986557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f1c03-e74a-4331-a3fd-7668ceaf668f None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:48.010169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:48.021387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341e601-6d03-47e7-88dc-41049a641a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:48.022002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341e601-6d03-47e7-88dc-41049a641a15 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:48.022310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341e601-6d03-47e7-88dc-41049a641a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:48.022581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341e601-6d03-47e7-88dc-41049a641a15 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:48.023225 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341e601-6d03-47e7-88dc-41049a641a15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:48.023533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341e601-6d03-47e7-88dc-41049a641a15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:48.024888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2341e601-6d03-47e7-88dc-41049a641a15 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:48.037522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 307/619] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:48.042866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341e601-6d03-47e7-88dc-41049a641a15 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:48.047292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 313/620] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:12:47 2022] POST /identity/v3/services => generated 269 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:48.047955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82542309-8545-4082-9dbf-8891808f4304 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:48.171790 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:48.183912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341e601-6d03-47e7-88dc-41049a641a15 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:48.217341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82542309-8545-4082-9dbf-8891808f4304 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:48.228733 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebab04-f846-4431-99a9-b6a0f0c85983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:48.229692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebab04-f846-4431-99a9-b6a0f0c85983 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:48.230286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebab04-f846-4431-99a9-b6a0f0c85983 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:48.230765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebab04-f846-4431-99a9-b6a0f0c85983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:48.245512 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebab04-f846-4431-99a9-b6a0f0c85983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:48.246534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebab04-f846-4431-99a9-b6a0f0c85983 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-6cebab04-f846-4431-99a9-b6a0f0c85983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:48.263634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:48.265133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:48.266007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:48.266007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:48.266007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:48.267230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05264acd-d1c4-47a2-8657-4094c0d58a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:48.267872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05264acd-d1c4-47a2-8657-4094c0d58a9c None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:48.268153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05264acd-d1c4-47a2-8657-4094c0d58a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:48.268440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05264acd-d1c4-47a2-8657-4094c0d58a9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:48.268711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 308/621] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:48.269665 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebab04-f846-4431-99a9-b6a0f0c85983 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:48.277174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05264acd-d1c4-47a2-8657-4094c0d58a9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:48.279058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05264acd-d1c4-47a2-8657-4094c0d58a9c None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-05264acd-d1c4-47a2-8657-4094c0d58a9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:48.291567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:48.292501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:48.293477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:48.293477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:48.293477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:48.293477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:48.293477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:48.293477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:48.293477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:48.293477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:48.293477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 314/622] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:48.293996 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05264acd-d1c4-47a2-8657-4094c0d58a9c None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:48.314660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:48.394672 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebab04-f846-4431-99a9-b6a0f0c85983 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:48.410761 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05264acd-d1c4-47a2-8657-4094c0d58a9c None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:48.441279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf3153-ac94-4ddf-bc5b-fa669d84ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:48.442592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf3153-ac94-4ddf-bc5b-fa669d84ec10 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:48.443355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf3153-ac94-4ddf-bc5b-fa669d84ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:48.443855 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf3153-ac94-4ddf-bc5b-fa669d84ec10 None admin] PATH_INFO: `/v3/services/network` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:48.445329 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf3153-ac94-4ddf-bc5b-fa669d84ec10 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:48.446590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:48.448472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf3153-ac94-4ddf-bc5b-fa669d84ec10 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:48.452644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cbb0f-da76-4347-92a6-757d9a4a0707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:48.453254 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cbb0f-da76-4347-92a6-757d9a4a0707 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:48.453526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cbb0f-da76-4347-92a6-757d9a4a0707 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:48.453849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cbb0f-da76-4347-92a6-757d9a4a0707 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:48.454659 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cbb0f-da76-4347-92a6-757d9a4a0707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:48.455020 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cbb0f-da76-4347-92a6-757d9a4a0707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:48.456413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322cbb0f-da76-4347-92a6-757d9a4a0707 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-08cf3153-ac94-4ddf-bc5b-fa669d84ec10 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:48.461384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:48.462448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 19 02:12:48.464496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:48.468330 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 309/623] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:48 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:48.469312 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf3153-ac94-4ddf-bc5b-fa669d84ec10 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:48.476078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 315/624] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:48.477083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cbb0f-da76-4347-92a6-757d9a4a0707 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:48.592459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:48.598571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf3153-ac94-4ddf-bc5b-fa669d84ec10 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:48.615170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cbb0f-da76-4347-92a6-757d9a4a0707 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:48.641055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d41da3-41cb-4e92-848a-34583ead65a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:48.642118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d41da3-41cb-4e92-848a-34583ead65a2 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:48.642729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d41da3-41cb-4e92-848a-34583ead65a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:48.643208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d41da3-41cb-4e92-848a-34583ead65a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:48.644413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d41da3-41cb-4e92-848a-34583ead65a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:48.644954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d41da3-41cb-4e92-848a-34583ead65a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:48.648818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d41da3-41cb-4e92-848a-34583ead65a2 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:48.667307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 310/625] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:48.668456 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d41da3-41cb-4e92-848a-34583ead65a2 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:48.679347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a629aad-20ae-4b48-a6b5-ef2a7e1e156c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:48.680065 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a629aad-20ae-4b48-a6b5-ef2a7e1e156c None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:48.680271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a629aad-20ae-4b48-a6b5-ef2a7e1e156c None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:48.680495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a629aad-20ae-4b48-a6b5-ef2a7e1e156c None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:48.681121 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a629aad-20ae-4b48-a6b5-ef2a7e1e156c None admin] RBAC: Adding query filter params (name=network) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:48.681418 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a629aad-20ae-4b48-a6b5-ef2a7e1e156c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:48.687594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a629aad-20ae-4b48-a6b5-ef2a7e1e156c None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:48.707491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 316/626] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:12:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:48.708362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a629aad-20ae-4b48-a6b5-ef2a7e1e156c None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:48.791476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d41da3-41cb-4e92-848a-34583ead65a2 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:48.808234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:48.849381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602946a-407e-4a68-88e1-6d709ac88885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:48.850658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602946a-407e-4a68-88e1-6d709ac88885 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:48.851172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602946a-407e-4a68-88e1-6d709ac88885 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:48.851987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602946a-407e-4a68-88e1-6d709ac88885 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:48.861060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602946a-407e-4a68-88e1-6d709ac88885 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:48.862851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602946a-407e-4a68-88e1-6d709ac88885 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-1602946a-407e-4a68-88e1-6d709ac88885 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:48.877398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:48.878413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:48.879341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:48.879341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:48.879341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:48.879341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:48.879341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:48.879341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:48.879341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:48.879341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:48.879341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 19 02:12:48.879341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:48.882780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 311/627] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:12:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:48.883778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602946a-407e-4a68-88e1-6d709ac88885 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:48.903016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a629aad-20ae-4b48-a6b5-ef2a7e1e156c None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:48.906338 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:48.973219 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a49e2-0524-4537-b1bb-6bf67a32d9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:48.973851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a49e2-0524-4537-b1bb-6bf67a32d9bd None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:48.974183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a49e2-0524-4537-b1bb-6bf67a32d9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:48.974494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a49e2-0524-4537-b1bb-6bf67a32d9bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:48.975224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a49e2-0524-4537-b1bb-6bf67a32d9bd None admin] RBAC: Adding query filter params (role.id=695bad5e383d4052a3d6ffa680329cb7, scope.project.id=676e986ff972410d9f498800d313476f, user.id=717c74cb769148a5b465079add961892) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:48.975528 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a49e2-0524-4537-b1bb-6bf67a32d9bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:48.977030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a49e2-0524-4537-b1bb-6bf67a32d9bd None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:48.998403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 317/628] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:12:48 2022] GET /identity/v3/role_assignments?user.id=717c74cb769148a5b465079add961892&scope.project.id=676e986ff972410d9f498800d313476f&role.id=695bad5e383d4052a3d6ffa680329cb7 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:48.999335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a49e2-0524-4537-b1bb-6bf67a32d9bd None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:49.009702 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602946a-407e-4a68-88e1-6d709ac88885 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:49.058356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06610e-d1b6-40fa-9224-01f5bfaab3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:49.063101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06610e-d1b6-40fa-9224-01f5bfaab3b6 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:49.064294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06610e-d1b6-40fa-9224-01f5bfaab3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:49.064955 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06610e-d1b6-40fa-9224-01f5bfaab3b6 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:49.067901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06610e-d1b6-40fa-9224-01f5bfaab3b6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:49.068711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06610e-d1b6-40fa-9224-01f5bfaab3b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:49.073624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06610e-d1b6-40fa-9224-01f5bfaab3b6 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:49.091292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 312/629] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:12:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:49.109025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06610e-d1b6-40fa-9224-01f5bfaab3b6 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:49.277493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a49e2-0524-4537-b1bb-6bf67a32d9bd None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:49.290290 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:49.300149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:49.310427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06610e-d1b6-40fa-9224-01f5bfaab3b6 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:49.319768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba1944-21f3-4441-88ab-16259fa78ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:49.320372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba1944-21f3-4441-88ab-16259fa78ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:49.349829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba1944-21f3-4441-88ab-16259fa78ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:49.349829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba1944-21f3-4441-88ab-16259fa78ae1 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:49.349829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba1944-21f3-4441-88ab-16259fa78ae1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:49.349829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba1944-21f3-4441-88ab-16259fa78ae1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:49.349829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba1944-21f3-4441-88ab-16259fa78ae1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:49.349829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 318/630] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Oct 19 02:12:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:49.350299 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba1944-21f3-4441-88ab-16259fa78ae1 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:49.350711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca214a4d-3842-4239-b054-e8b0fc4d363e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:49.351326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca214a4d-3842-4239-b054-e8b0fc4d363e None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:49.351597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca214a4d-3842-4239-b054-e8b0fc4d363e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:49.351858 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca214a4d-3842-4239-b054-e8b0fc4d363e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:49.352526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca214a4d-3842-4239-b054-e8b0fc4d363e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:49.353881 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca214a4d-3842-4239-b054-e8b0fc4d363e None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:49.432497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 313/631] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:12:49 2022] POST /identity/v3/endpoints => generated 337 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:49.483625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba1944-21f3-4441-88ab-16259fa78ae1 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:49.538164 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce1512-9606-4086-9e64-decb707b655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:49.538816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1512-9606-4086-9e64-decb707b655f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:49.539092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1512-9606-4086-9e64-decb707b655f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:49.539358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce1512-9606-4086-9e64-decb707b655f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:49.610472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:49.645560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce1512-9606-4086-9e64-decb707b655f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:49.647053 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce1512-9606-4086-9e64-decb707b655f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:49.800393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 319/632] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:49 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 02:12:49.805515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca214a4d-3842-4239-b054-e8b0fc4d363e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:49.910845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca214a4d-3842-4239-b054-e8b0fc4d363e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:49.948886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b35439-1667-47f6-b318-875eed8c712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:49.949523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b35439-1667-47f6-b318-875eed8c712a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:49.949793 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b35439-1667-47f6-b318-875eed8c712a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:49.950088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b35439-1667-47f6-b318-875eed8c712a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:49.958147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:49.961963 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b35439-1667-47f6-b318-875eed8c712a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:49.963440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b35439-1667-47f6-b318-875eed8c712a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-06b35439-1667-47f6-b318-875eed8c712a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:49.978464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:49.979461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:49.980353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:49.980353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:49.980353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:49.980353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 314/633] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:49.985236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce1512-9606-4086-9e64-decb707b655f None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:50.052160 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9f985-8bf9-4ed8-baee-2a16f68c1785 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:50.052345 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9f985-8bf9-4ed8-baee-2a16f68c1785 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:50.052684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e9f985-8bf9-4ed8-baee-2a16f68c1785 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:50.054368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 315/634] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:50.063612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a58b8-0ee8-4a19-be3f-b2ca2d2046f1 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:50.064021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a58b8-0ee8-4a19-be3f-b2ca2d2046f1 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:50.064392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a58b8-0ee8-4a19-be3f-b2ca2d2046f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:50.068533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:50.087841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2d9a58b8-0ee8-4a19-be3f-b2ca2d2046f1 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:50.090025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2d9a58b8-0ee8-4a19-be3f-b2ca2d2046f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:50.108893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce1512-9606-4086-9e64-decb707b655f None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:50.147847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbadec8-bf71-4779-8c11-9f34b462f358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:50.148453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbadec8-bf71-4779-8c11-9f34b462f358 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:50.148731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbadec8-bf71-4779-8c11-9f34b462f358 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:50.149001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbadec8-bf71-4779-8c11-9f34b462f358 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:50.149642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbadec8-bf71-4779-8c11-9f34b462f358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:50.149942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbadec8-bf71-4779-8c11-9f34b462f358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:50.151324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbadec8-bf71-4779-8c11-9f34b462f358 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:50.164177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 320/635] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:50.169562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbadec8-bf71-4779-8c11-9f34b462f358 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:50.186198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 316/636] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:50 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:12:50.272298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbadec8-bf71-4779-8c11-9f34b462f358 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:50.302424 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:50.307314 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb751b-5627-46fd-89ae-d70213f4da6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:50.307964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb751b-5627-46fd-89ae-d70213f4da6c None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:50.308238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb751b-5627-46fd-89ae-d70213f4da6c None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:50.308511 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb751b-5627-46fd-89ae-d70213f4da6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:50.316328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb751b-5627-46fd-89ae-d70213f4da6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:50.317818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb751b-5627-46fd-89ae-d70213f4da6c None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-76fb751b-5627-46fd-89ae-d70213f4da6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:50.330877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:50.331833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:50.332716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:50.332716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:50.332716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:50.332716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:50.332716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:50.332716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:50.332716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:50.332716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:50.332716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 321/637] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:50.338113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a58b8-0ee8-4a19-be3f-b2ca2d2046f1 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:50.418412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3c855-3dff-4f07-b5ae-7b0a046c5bd4 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:50.418730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3c855-3dff-4f07-b5ae-7b0a046c5bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:50.418968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3c855-3dff-4f07-b5ae-7b0a046c5bd4 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:50.420588 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 322/638] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:12:50.426691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05dea8-2ebb-487f-ba5d-c0a9796e2425 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:50.427083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05dea8-2ebb-487f-ba5d-c0a9796e2425 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:50.427460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05dea8-2ebb-487f-ba5d-c0a9796e2425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:50.449496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-6b05dea8-2ebb-487f-ba5d-c0a9796e2425 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:50.450832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a58b8-0ee8-4a19-be3f-b2ca2d2046f1 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:50.451285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6b05dea8-2ebb-487f-ba5d-c0a9796e2425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:50.471953 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:50.488094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb0cef-080c-4e1f-a5a2-d460f338ef39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:50.494836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb0cef-080c-4e1f-a5a2-d460f338ef39 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:50.494836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb0cef-080c-4e1f-a5a2-d460f338ef39 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:50.494836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb0cef-080c-4e1f-a5a2-d460f338ef39 None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:50.494836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb0cef-080c-4e1f-a5a2-d460f338ef39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:50.494836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb0cef-080c-4e1f-a5a2-d460f338ef39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:50.494836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb0cef-080c-4e1f-a5a2-d460f338ef39 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:50.503571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 317/639] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:50.504463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb0cef-080c-4e1f-a5a2-d460f338ef39 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:50.533632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 323/640] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:50 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:12:50.534801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05dea8-2ebb-487f-ba5d-c0a9796e2425 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:50.627865 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb0cef-080c-4e1f-a5a2-d460f338ef39 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:50.664561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05dea8-2ebb-487f-ba5d-c0a9796e2425 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:50.665488 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05970b0d-4a8d-4b5d-8958-c587b1f81a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:50.666319 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05970b0d-4a8d-4b5d-8958-c587b1f81a7e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:50.666604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05970b0d-4a8d-4b5d-8958-c587b1f81a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:50.666880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05970b0d-4a8d-4b5d-8958-c587b1f81a7e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:50.667474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05970b0d-4a8d-4b5d-8958-c587b1f81a7e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:50.668846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05970b0d-4a8d-4b5d-8958-c587b1f81a7e None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-05970b0d-4a8d-4b5d-8958-c587b1f81a7e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:50.677880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:50.686977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:50.687789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 318/641] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:12:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:50.688624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05970b0d-4a8d-4b5d-8958-c587b1f81a7e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:50.694454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:50.706247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b33cf2-3a3a-42af-b903-f76000a799b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:50.707076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b33cf2-3a3a-42af-b903-f76000a799b4 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:50.707452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b33cf2-3a3a-42af-b903-f76000a799b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:50.707859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b33cf2-3a3a-42af-b903-f76000a799b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:50.708691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b33cf2-3a3a-42af-b903-f76000a799b4 None admin] RBAC: Adding query filter params (role.id=d5070bb151d343ca8de32abe373f4a73, scope.project.id=676e986ff972410d9f498800d313476f, user.id=e476a32b458c4ea0bdc207d0ff4bfd47) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:50.709096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b33cf2-3a3a-42af-b903-f76000a799b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:50.710836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b33cf2-3a3a-42af-b903-f76000a799b4 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:50.718148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 324/642] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:12:50 2022] GET /identity/v3/role_assignments?user.id=e476a32b458c4ea0bdc207d0ff4bfd47&scope.project.id=676e986ff972410d9f498800d313476f&role.id=d5070bb151d343ca8de32abe373f4a73 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:50.747530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:50.805882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05970b0d-4a8d-4b5d-8958-c587b1f81a7e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:50.807425 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b33cf2-3a3a-42af-b903-f76000a799b4 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:50.843820 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bd314-51ff-4698-8d0c-d3b1b2b8e05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:50.844439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bd314-51ff-4698-8d0c-d3b1b2b8e05c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:50.844715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bd314-51ff-4698-8d0c-d3b1b2b8e05c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:50.844988 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bd314-51ff-4698-8d0c-d3b1b2b8e05c None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:50.845693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bd314-51ff-4698-8d0c-d3b1b2b8e05c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:50.846003 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bd314-51ff-4698-8d0c-d3b1b2b8e05c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:50.847497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bd314-51ff-4698-8d0c-d3b1b2b8e05c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:50.853629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 319/643] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:12:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:50.859392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bd314-51ff-4698-8d0c-d3b1b2b8e05c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:50.931070 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b33cf2-3a3a-42af-b903-f76000a799b4 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:50.968829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9853e7-b91d-4c90-94ce-b47d772dbd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:50.969472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9853e7-b91d-4c90-94ce-b47d772dbd53 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:50.969754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9853e7-b91d-4c90-94ce-b47d772dbd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:50.970000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9853e7-b91d-4c90-94ce-b47d772dbd53 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:50.971502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9853e7-b91d-4c90-94ce-b47d772dbd53 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:50.973075 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9853e7-b91d-4c90-94ce-b47d772dbd53 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-bc9853e7-b91d-4c90-94ce-b47d772dbd53 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:50.983519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:50.984444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:50.985322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 325/644] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Oct 19 02:12:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:50.989969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bd314-51ff-4698-8d0c-d3b1b2b8e05c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:50.992662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9853e7-b91d-4c90-94ce-b47d772dbd53 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:51.000540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:51.028899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fec18b-0945-4a11-adbb-e273d12a2bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:51.029832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fec18b-0945-4a11-adbb-e273d12a2bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:51.030742 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fec18b-0945-4a11-adbb-e273d12a2bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:51.031414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fec18b-0945-4a11-adbb-e273d12a2bd0 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:51.032417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fec18b-0945-4a11-adbb-e273d12a2bd0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:51.032876 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fec18b-0945-4a11-adbb-e273d12a2bd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:51.034359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fec18b-0945-4a11-adbb-e273d12a2bd0 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:51.040035 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 320/645] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:12:50 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:51.055461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:51.115591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9853e7-b91d-4c90-94ce-b47d772dbd53 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:51.154505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b434bd-366a-4017-9e1b-65d16f47c313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:51.155237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b434bd-366a-4017-9e1b-65d16f47c313 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:51.155517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b434bd-366a-4017-9e1b-65d16f47c313 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:51.155793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b434bd-366a-4017-9e1b-65d16f47c313 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:51.156431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b434bd-366a-4017-9e1b-65d16f47c313 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:51.156731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b434bd-366a-4017-9e1b-65d16f47c313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:51.158192 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b434bd-366a-4017-9e1b-65d16f47c313 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:51.168386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 326/646] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 19 02:12:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:51.169767 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fec18b-0945-4a11-adbb-e273d12a2bd0 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:51.278784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fec18b-0945-4a11-adbb-e273d12a2bd0 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:51.313637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23de9f-7f20-4c0f-8c40-797492acb007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:51.314263 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23de9f-7f20-4c0f-8c40-797492acb007 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:51.314537 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23de9f-7f20-4c0f-8c40-797492acb007 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:51.314800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23de9f-7f20-4c0f-8c40-797492acb007 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:51.315419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23de9f-7f20-4c0f-8c40-797492acb007 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:51.315714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23de9f-7f20-4c0f-8c40-797492acb007 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:51.317066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23de9f-7f20-4c0f-8c40-797492acb007 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:51.322351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 321/647] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 19 02:12:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:51.328752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b434bd-366a-4017-9e1b-65d16f47c313 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:51.432250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b434bd-366a-4017-9e1b-65d16f47c313 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:51.449282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:51.471886 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d9deb-86dd-4159-978b-aaabf7f9747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:51.472508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d9deb-86dd-4159-978b-aaabf7f9747f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:51.472791 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d9deb-86dd-4159-978b-aaabf7f9747f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:51.473069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d9deb-86dd-4159-978b-aaabf7f9747f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:51.473786 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d9deb-86dd-4159-978b-aaabf7f9747f None admin] RBAC: Adding query filter params (interface=public, service_id=3c943040ddb4457abc3f5525faab7778, region_id=RegionOne) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:51.474120 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d9deb-86dd-4159-978b-aaabf7f9747f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:51.475504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d9deb-86dd-4159-978b-aaabf7f9747f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:51.481404 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 327/648] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 19 02:12:51 2022] GET /identity/v3/endpoints?service_id=3c943040ddb4457abc3f5525faab7778&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:51.487099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23de9f-7f20-4c0f-8c40-797492acb007 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:51.526966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:51.558669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c88d-0de4-47a2-8a61-1479237ddbd0 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:51.559083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c88d-0de4-47a2-8a61-1479237ddbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:51.559516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c88d-0de4-47a2-8a61-1479237ddbd0 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:51.561151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 328/649] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:51.566823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfdb73-195d-431b-bc58-8e9cbf696441 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:51.567214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfdb73-195d-431b-bc58-8e9cbf696441 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:51.567594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfdb73-195d-431b-bc58-8e9cbf696441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:51.589284 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c0bfdb73-195d-431b-bc58-8e9cbf696441 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:51.590674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c0bfdb73-195d-431b-bc58-8e9cbf696441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:51.598408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23de9f-7f20-4c0f-8c40-797492acb007 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:51.636188 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcbc95-c60e-49ba-8b5d-c8812647fdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:51.637124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcbc95-c60e-49ba-8b5d-c8812647fdba None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:51.637606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcbc95-c60e-49ba-8b5d-c8812647fdba None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:51.638100 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcbc95-c60e-49ba-8b5d-c8812647fdba None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:51.638882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcbc95-c60e-49ba-8b5d-c8812647fdba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:51.640514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcbc95-c60e-49ba-8b5d-c8812647fdba None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:51.646646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 322/650] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 19 02:12:51 2022] GET /identity/v3/services => generated 1029 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:12:51.675033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 329/651] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:51 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:51.944533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcbc95-c60e-49ba-8b5d-c8812647fdba None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:52.047563 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:52.050676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcbc95-c60e-49ba-8b5d-c8812647fdba None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:52.084413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b7342-fd5a-4b53-941e-aa3c050f393e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:52.085044 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b7342-fd5a-4b53-941e-aa3c050f393e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:52.085324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b7342-fd5a-4b53-941e-aa3c050f393e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:52.085599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b7342-fd5a-4b53-941e-aa3c050f393e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:52.096860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b7342-fd5a-4b53-941e-aa3c050f393e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-df2b7342-fd5a-4b53-941e-aa3c050f393e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:52.102218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:52.103153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:52.104370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 323/652] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 19 02:12:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:52.110524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfdb73-195d-431b-bc58-8e9cbf696441 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:52.218695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfdb73-195d-431b-bc58-8e9cbf696441 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:52.228058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:52.253087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf670bba-11a5-4e70-8911-9eab8cb906ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:52.253696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf670bba-11a5-4e70-8911-9eab8cb906ba None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:52.253978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf670bba-11a5-4e70-8911-9eab8cb906ba None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:52.254282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf670bba-11a5-4e70-8911-9eab8cb906ba None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:52.254940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf670bba-11a5-4e70-8911-9eab8cb906ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:52.255245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf670bba-11a5-4e70-8911-9eab8cb906ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:52.256609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf670bba-11a5-4e70-8911-9eab8cb906ba None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:52.265092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 330/653] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Oct 19 02:12:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:52.270585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b7342-fd5a-4b53-941e-aa3c050f393e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:52.372593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b7342-fd5a-4b53-941e-aa3c050f393e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:52.407606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88619068-0064-4929-ab48-2b743943264d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:52.408243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88619068-0064-4929-ab48-2b743943264d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:52.408522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88619068-0064-4929-ab48-2b743943264d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:52.408791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88619068-0064-4929-ab48-2b743943264d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:52.419640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88619068-0064-4929-ab48-2b743943264d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:52.421172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88619068-0064-4929-ab48-2b743943264d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-88619068-0064-4929-ab48-2b743943264d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:52.435726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:52.437211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:52.438523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:52.438523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:52.438523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:52.438523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 324/654] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:52.443234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf670bba-11a5-4e70-8911-9eab8cb906ba None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:52.546464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf670bba-11a5-4e70-8911-9eab8cb906ba None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:52.581735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5bca0-f7fa-431d-b01f-dc1d6fa979e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:52.582388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5bca0-f7fa-431d-b01f-dc1d6fa979e1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:52.582668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5bca0-f7fa-431d-b01f-dc1d6fa979e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:52.582942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5bca0-f7fa-431d-b01f-dc1d6fa979e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:52.583599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5bca0-f7fa-431d-b01f-dc1d6fa979e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:52.583903 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5bca0-f7fa-431d-b01f-dc1d6fa979e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:52.585297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5bca0-f7fa-431d-b01f-dc1d6fa979e1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:52.599034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 331/655] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:52.604515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88619068-0064-4929-ab48-2b743943264d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:52.622835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:52.710101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88619068-0064-4929-ab48-2b743943264d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:52.744634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb7da8-4f7c-4dbb-99bd-8357960b4dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:52.745264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb7da8-4f7c-4dbb-99bd-8357960b4dda None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:52.745541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb7da8-4f7c-4dbb-99bd-8357960b4dda None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:52.745812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb7da8-4f7c-4dbb-99bd-8357960b4dda None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:52.753523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb7da8-4f7c-4dbb-99bd-8357960b4dda None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:52.754987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb7da8-4f7c-4dbb-99bd-8357960b4dda None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-f5cb7da8-4f7c-4dbb-99bd-8357960b4dda None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:52.766885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:52.767852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:52.769388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:52.769388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:52.769388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:52.769388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:52.769388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:52.769388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:52.769388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:52.769388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:52.769388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 02:12:52.769388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:52.769388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 325/656] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 19 02:12:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:52.773966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5bca0-f7fa-431d-b01f-dc1d6fa979e1 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:52.805075 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:52.876892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5bca0-f7fa-431d-b01f-dc1d6fa979e1 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:52.912389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc2388-ab8c-4727-8b8e-b19aa91277cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:52.912995 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc2388-ab8c-4727-8b8e-b19aa91277cb None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:52.913274 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc2388-ab8c-4727-8b8e-b19aa91277cb None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:52.913548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc2388-ab8c-4727-8b8e-b19aa91277cb None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:52.914310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc2388-ab8c-4727-8b8e-b19aa91277cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:52.914617 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc2388-ab8c-4727-8b8e-b19aa91277cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:52.916039 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc2388-ab8c-4727-8b8e-b19aa91277cb None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:52.936859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 332/657] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Oct 19 02:12:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:52.942515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb7da8-4f7c-4dbb-99bd-8357960b4dda None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:53.017859 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:53.045285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d67bb4-996a-4607-b402-ff9c9a338ad2 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.045979 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d67bb4-996a-4607-b402-ff9c9a338ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.046513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d67bb4-996a-4607-b402-ff9c9a338ad2 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.048238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 333/658] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:12:53.049457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb7da8-4f7c-4dbb-99bd-8357960b4dda None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:53.055600 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653d91f-4021-4474-b323-fc87f3c66cf9 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.056118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653d91f-4021-4474-b323-fc87f3c66cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.057193 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653d91f-4021-4474-b323-fc87f3c66cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.079584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-d653d91f-4021-4474-b323-fc87f3c66cf9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:53.081721 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d653d91f-4021-4474-b323-fc87f3c66cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:53.085729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1eaf56-373e-4de5-b560-1d0b2858668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:53.085729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eaf56-373e-4de5-b560-1d0b2858668b None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.086106 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eaf56-373e-4de5-b560-1d0b2858668b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.086273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1eaf56-373e-4de5-b560-1d0b2858668b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.107891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1eaf56-373e-4de5-b560-1d0b2858668b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:53.109318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1eaf56-373e-4de5-b560-1d0b2858668b None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-de1eaf56-373e-4de5-b560-1d0b2858668b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:53.126141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:53.127095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:53.128408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:53.128408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:53.128408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:53.128408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 326/659] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:53.132692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1eaf56-373e-4de5-b560-1d0b2858668b None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:53.170196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 334/660] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:53 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:12:53.240660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1eaf56-373e-4de5-b560-1d0b2858668b None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:53.264876 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:53.275957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59502566-a59c-40b6-9f4c-1715136332e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:53.276567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59502566-a59c-40b6-9f4c-1715136332e7 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.281454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59502566-a59c-40b6-9f4c-1715136332e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.281454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59502566-a59c-40b6-9f4c-1715136332e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.281454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59502566-a59c-40b6-9f4c-1715136332e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:53.281454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59502566-a59c-40b6-9f4c-1715136332e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:53.281454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59502566-a59c-40b6-9f4c-1715136332e7 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:53.292041 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 327/661] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:53.298092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653d91f-4021-4474-b323-fc87f3c66cf9 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:53.313079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec50016-d520-43c4-8a7d-7c0d7f86ebeb None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.313346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec50016-d520-43c4-8a7d-7c0d7f86ebeb None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.313547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec50016-d520-43c4-8a7d-7c0d7f86ebeb None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.315168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 328/662] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:12:53.319913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5cc72-17d1-48a0-934b-0b5649d362d2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.320192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5cc72-17d1-48a0-934b-0b5649d362d2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.320464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5cc72-17d1-48a0-934b-0b5649d362d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.339653 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9aa5cc72-17d1-48a0-934b-0b5649d362d2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:53.340820 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9aa5cc72-17d1-48a0-934b-0b5649d362d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:53.350863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:53.407490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653d91f-4021-4474-b323-fc87f3c66cf9 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:53.434148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 329/663] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:53 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:12:53.450802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5cc72-17d1-48a0-934b-0b5649d362d2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:53.451868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906def9b-576d-45bc-bb34-378df9a5445e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:53.453106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906def9b-576d-45bc-bb34-378df9a5445e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.453386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906def9b-576d-45bc-bb34-378df9a5445e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.453654 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906def9b-576d-45bc-bb34-378df9a5445e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.460661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906def9b-576d-45bc-bb34-378df9a5445e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:53.462154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906def9b-576d-45bc-bb34-378df9a5445e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-906def9b-576d-45bc-bb34-378df9a5445e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:53.474716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:53.475624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:53.476538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:53.476538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:53.476538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:53.476538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:53.476538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:53.476538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:53.476538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:53.476538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:53.480502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 335/664] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:53.481669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906def9b-576d-45bc-bb34-378df9a5445e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:53.566472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:53.576678 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5cc72-17d1-48a0-934b-0b5649d362d2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:53.589022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906def9b-576d-45bc-bb34-378df9a5445e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:53.594275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:53.617100 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd89984-ae3d-46ef-90bf-ad51e9e64706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:53.618508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd89984-ae3d-46ef-90bf-ad51e9e64706 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.618996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd89984-ae3d-46ef-90bf-ad51e9e64706 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.619634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd89984-ae3d-46ef-90bf-ad51e9e64706 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.620746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd89984-ae3d-46ef-90bf-ad51e9e64706 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:53.622800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd89984-ae3d-46ef-90bf-ad51e9e64706 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:53.625349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f3826-e82c-4a5d-b517-91b825817411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:53.626151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f3826-e82c-4a5d-b517-91b825817411 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.626396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f3826-e82c-4a5d-b517-91b825817411 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.626719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f3826-e82c-4a5d-b517-91b825817411 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.627408 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f3826-e82c-4a5d-b517-91b825817411 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:53.627797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f3826-e82c-4a5d-b517-91b825817411 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:53.629123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f3826-e82c-4a5d-b517-91b825817411 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-4dd89984-ae3d-46ef-90bf-ad51e9e64706 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:53.634404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:53.635439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:12:53.636264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:53.644080 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 330/666] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 19 02:12:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:53.646998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd89984-ae3d-46ef-90bf-ad51e9e64706 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:53.647419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 336/666] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:53.648690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f3826-e82c-4a5d-b517-91b825817411 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:53.761275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd89984-ae3d-46ef-90bf-ad51e9e64706 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:53.761505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f3826-e82c-4a5d-b517-91b825817411 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:53.794875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297e172-cdb0-42e3-b9c1-d139e29160d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:53.795107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb7464-c032-4f99-979d-63ff5f0e56b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:53.795686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297e172-cdb0-42e3-b9c1-d139e29160d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.795826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb7464-c032-4f99-979d-63ff5f0e56b0 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.796009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297e172-cdb0-42e3-b9c1-d139e29160d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.796103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb7464-c032-4f99-979d-63ff5f0e56b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.796258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297e172-cdb0-42e3-b9c1-d139e29160d5 None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/717c74cb769148a5b465079add961892/roles/e95a043c68e440b2a961391d6e6e625b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.796361 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb7464-c032-4f99-979d-63ff5f0e56b0 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.796987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb7464-c032-4f99-979d-63ff5f0e56b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:53.797353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb7464-c032-4f99-979d-63ff5f0e56b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:53.798573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297e172-cdb0-42e3-b9c1-d139e29160d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=717c74cb769148a5b465079add961892, role_id=e95a043c68e440b2a961391d6e6e625b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:53.798757 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb7464-c032-4f99-979d-63ff5f0e56b0 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:53.800032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297e172-cdb0-42e3-b9c1-d139e29160d5 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:53.804124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 331/667] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:12:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:53.804962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb7464-c032-4f99-979d-63ff5f0e56b0 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:53.854350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:53.868532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:53.874612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 337/668] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:12:53 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/717c74cb769148a5b465079add961892/roles/e95a043c68e440b2a961391d6e6e625b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:12:53.882132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297e172-cdb0-42e3-b9c1-d139e29160d5 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:53.921391 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb7464-c032-4f99-979d-63ff5f0e56b0 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:53.961101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2309a1-4cba-4b5b-b474-4ffa6afa5025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:53.961719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2309a1-4cba-4b5b-b474-4ffa6afa5025 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:53.961995 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2309a1-4cba-4b5b-b474-4ffa6afa5025 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:53.962293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2309a1-4cba-4b5b-b474-4ffa6afa5025 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:53.962960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2309a1-4cba-4b5b-b474-4ffa6afa5025 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:53.964583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2309a1-4cba-4b5b-b474-4ffa6afa5025 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:53.999832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297e172-cdb0-42e3-b9c1-d139e29160d5 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:54.036690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3c6ca0-18a8-458f-96c9-9bc82c9a3ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:54.037312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c6ca0-18a8-458f-96c9-9bc82c9a3ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:54.037593 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c6ca0-18a8-458f-96c9-9bc82c9a3ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:54.037866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3c6ca0-18a8-458f-96c9-9bc82c9a3ee8 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:54.038533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c6ca0-18a8-458f-96c9-9bc82c9a3ee8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:54.038840 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c6ca0-18a8-458f-96c9-9bc82c9a3ee8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:54.040221 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3c6ca0-18a8-458f-96c9-9bc82c9a3ee8 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:54.045191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 338/669] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:12:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:54.046926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd20885-7673-46a7-963f-b6ee47fb3fc0 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:54.047211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd20885-7673-46a7-963f-b6ee47fb3fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:54.047484 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd20885-7673-46a7-963f-b6ee47fb3fc0 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:54.048750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 339/670] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:54.054817 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07b6b5-dd96-4cda-9c4d-0792de17631a None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:54.055136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07b6b5-dd96-4cda-9c4d-0792de17631a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:54.055382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07b6b5-dd96-4cda-9c4d-0792de17631a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:54.065576 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 332/671] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:12:53 2022] POST /identity/v3/services => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:54.077176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-1c07b6b5-dd96-4cda-9c4d-0792de17631a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:54.078802 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1c07b6b5-dd96-4cda-9c4d-0792de17631a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:54.164167 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:54.183347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 340/672] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:12:54.456041 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2309a1-4cba-4b5b-b474-4ffa6afa5025 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:54.564153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2309a1-4cba-4b5b-b474-4ffa6afa5025 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:54.566142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:54.599850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de8f0f-5ca4-48aa-8592-c262c5295157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:54.600861 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de8f0f-5ca4-48aa-8592-c262c5295157 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:54.601600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de8f0f-5ca4-48aa-8592-c262c5295157 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:54.602643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de8f0f-5ca4-48aa-8592-c262c5295157 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:54.603460 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de8f0f-5ca4-48aa-8592-c262c5295157 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:54.605029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de8f0f-5ca4-48aa-8592-c262c5295157 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-a7de8f0f-5ca4-48aa-8592-c262c5295157 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:54.614589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:54.615530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 19 02:12:54.616403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:54.619128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 333/673] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Oct 19 02:12:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:54.627173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07b6b5-dd96-4cda-9c4d-0792de17631a None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:54.732577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07b6b5-dd96-4cda-9c4d-0792de17631a None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:54.769484 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06ba1c-8eaf-47ae-97f8-6fb9f0aea57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:54.770505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06ba1c-8eaf-47ae-97f8-6fb9f0aea57b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:54.770992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06ba1c-8eaf-47ae-97f8-6fb9f0aea57b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:54.771467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06ba1c-8eaf-47ae-97f8-6fb9f0aea57b None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:54.772304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06ba1c-8eaf-47ae-97f8-6fb9f0aea57b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:54.772830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06ba1c-8eaf-47ae-97f8-6fb9f0aea57b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:54.774432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06ba1c-8eaf-47ae-97f8-6fb9f0aea57b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:54.780575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 341/674] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 19 02:12:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:54.786456 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de8f0f-5ca4-48aa-8592-c262c5295157 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:54.894917 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de8f0f-5ca4-48aa-8592-c262c5295157 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:54.930197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1cbb1-b522-407b-a78f-9e02eff3ed6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:54.931118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1cbb1-b522-407b-a78f-9e02eff3ed6f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:54.931604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1cbb1-b522-407b-a78f-9e02eff3ed6f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:54.932079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1cbb1-b522-407b-a78f-9e02eff3ed6f None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:54.932907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1cbb1-b522-407b-a78f-9e02eff3ed6f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:54.933436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1cbb1-b522-407b-a78f-9e02eff3ed6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:54.935031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e1cbb1-b522-407b-a78f-9e02eff3ed6f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:54.940550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 334/675] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 19 02:12:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:54.946473 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06ba1c-8eaf-47ae-97f8-6fb9f0aea57b None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:55.009416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:55.056984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06ba1c-8eaf-47ae-97f8-6fb9f0aea57b None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:55.090803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787366c-f50a-4cbb-9aea-d9a1597ecea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:55.091398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787366c-f50a-4cbb-9aea-d9a1597ecea2 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:55.091672 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787366c-f50a-4cbb-9aea-d9a1597ecea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:55.091938 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787366c-f50a-4cbb-9aea-d9a1597ecea2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:55.092659 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787366c-f50a-4cbb-9aea-d9a1597ecea2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:55.094016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787366c-f50a-4cbb-9aea-d9a1597ecea2 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:55.188674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 342/676] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:12:54 2022] POST /identity/v3/endpoints => generated 347 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:56.167558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa1c65-b885-4b47-acb4-64aaba17738a None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:56.168097 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa1c65-b885-4b47-acb4-64aaba17738a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:56.168097 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa1c65-b885-4b47-acb4-64aaba17738a None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:56.169552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 335/677] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:56.175390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22206f-6d6b-4ed2-9519-4a294fbe9797 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:56.176177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22206f-6d6b-4ed2-9519-4a294fbe9797 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:56.176712 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c22206f-6d6b-4ed2-9519-4a294fbe9797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:56.198193 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-2c22206f-6d6b-4ed2-9519-4a294fbe9797 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:56.199738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2c22206f-6d6b-4ed2-9519-4a294fbe9797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:56.245278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a111f53-7e65-40b8-896b-cafb0381f3a3 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:56.245974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a111f53-7e65-40b8-896b-cafb0381f3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:56.246490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a111f53-7e65-40b8-896b-cafb0381f3a3 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:56.248231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 336/678] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:12:56.253293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0360e-1067-4d33-8326-d74c1bd7c70f None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:56.253813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0360e-1067-4d33-8326-d74c1bd7c70f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:56.254326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0360e-1067-4d33-8326-d74c1bd7c70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:56.277105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c5d0360e-1067-4d33-8326-d74c1bd7c70f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:56.278330 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c5d0360e-1067-4d33-8326-d74c1bd7c70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:56.282148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 343/679] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:56.306418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:56.313580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b8194-0eba-45e5-8495-d665efdb83b6 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:56.314035 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b8194-0eba-45e5-8495-d665efdb83b6 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:56.314575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b8194-0eba-45e5-8495-d665efdb83b6 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:56.317318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 344/680] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:12:56.321395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c106f4-5a83-470a-8ab2-0e2916cecc3d None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:56.321833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c106f4-5a83-470a-8ab2-0e2916cecc3d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:56.322277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c106f4-5a83-470a-8ab2-0e2916cecc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:56.341792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-97c106f4-5a83-470a-8ab2-0e2916cecc3d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:56.343785 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-97c106f4-5a83-470a-8ab2-0e2916cecc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:56.353817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 337/681] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:12:56.376266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:56.426229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 345/682] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:56.552825 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0360e-1067-4d33-8326-d74c1bd7c70f None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:56.619537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c106f4-5a83-470a-8ab2-0e2916cecc3d None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:56.662207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0360e-1067-4d33-8326-d74c1bd7c70f None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:56.700976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea8f39-c427-4632-aa1f-292aafe3d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:56.701727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea8f39-c427-4632-aa1f-292aafe3d06b None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:56.702063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea8f39-c427-4632-aa1f-292aafe3d06b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:56.702368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea8f39-c427-4632-aa1f-292aafe3d06b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:56.714598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea8f39-c427-4632-aa1f-292aafe3d06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-58ea8f39-c427-4632-aa1f-292aafe3d06b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:12:56.720432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:56.721346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:56.722193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 338/683] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 19 02:12:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:56.722882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea8f39-c427-4632-aa1f-292aafe3d06b None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:56.730618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c106f4-5a83-470a-8ab2-0e2916cecc3d None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:56.744300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:56.768208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1f8d9-8cd2-4cfc-88ad-08ee76914a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:56.768803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1f8d9-8cd2-4cfc-88ad-08ee76914a79 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:56.769081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1f8d9-8cd2-4cfc-88ad-08ee76914a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:56.769391 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1f8d9-8cd2-4cfc-88ad-08ee76914a79 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:56.770058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1f8d9-8cd2-4cfc-88ad-08ee76914a79 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:56.771434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1f8d9-8cd2-4cfc-88ad-08ee76914a79 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:56.821320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea8f39-c427-4632-aa1f-292aafe3d06b None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:56.843809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 346/684] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:12:56 2022] POST /identity/v3/services => generated 241 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:12:56.845546 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1f8d9-8cd2-4cfc-88ad-08ee76914a79 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:56.874511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1201a9-db50-45e0-860d-9bb85df1b571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:56.875906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1201a9-db50-45e0-860d-9bb85df1b571 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:56.876901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1201a9-db50-45e0-860d-9bb85df1b571 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:56.877439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1201a9-db50-45e0-860d-9bb85df1b571 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:56.878261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1201a9-db50-45e0-860d-9bb85df1b571 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:56.880621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1201a9-db50-45e0-860d-9bb85df1b571 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:56.885207 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-ef1201a9-db50-45e0-860d-9bb85df1b571 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:56.893910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:56.898013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:56.899030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 339/685] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:12:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:56.902666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1201a9-db50-45e0-860d-9bb85df1b571 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:57.002894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:57.006619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1f8d9-8cd2-4cfc-88ad-08ee76914a79 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:57.025538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1201a9-db50-45e0-860d-9bb85df1b571 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:57.054438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceea63-dc9b-4d39-beba-c81be1309115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:57.055357 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceea63-dc9b-4d39-beba-c81be1309115 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:57.055635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceea63-dc9b-4d39-beba-c81be1309115 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:57.055908 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceea63-dc9b-4d39-beba-c81be1309115 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:57.056572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ceea63-dc9b-4d39-beba-c81be1309115 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:57.056873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ceea63-dc9b-4d39-beba-c81be1309115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:57.058780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ceea63-dc9b-4d39-beba-c81be1309115 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:57.069751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 347/686] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Oct 19 02:12:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:57.074897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceea63-dc9b-4d39-beba-c81be1309115 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:57.079366 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a849423-813e-4b0c-8e90-f5f5ca3b8241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:57.080933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a849423-813e-4b0c-8e90-f5f5ca3b8241 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:57.081217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a849423-813e-4b0c-8e90-f5f5ca3b8241 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:57.081492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a849423-813e-4b0c-8e90-f5f5ca3b8241 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:57.083086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a849423-813e-4b0c-8e90-f5f5ca3b8241 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:57.083086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a849423-813e-4b0c-8e90-f5f5ca3b8241 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:57.083811 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a849423-813e-4b0c-8e90-f5f5ca3b8241 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:57.090508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 340/687] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:12:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:57.096538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a849423-813e-4b0c-8e90-f5f5ca3b8241 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:57.194091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:57.217411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceea63-dc9b-4d39-beba-c81be1309115 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:57.218820 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a849423-813e-4b0c-8e90-f5f5ca3b8241 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:57.254384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f114ee-50e4-4dd0-9ba2-b6f0c074db61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:57.254997 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f114ee-50e4-4dd0-9ba2-b6f0c074db61 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:57.255272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f114ee-50e4-4dd0-9ba2-b6f0c074db61 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:57.255546 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f114ee-50e4-4dd0-9ba2-b6f0c074db61 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:57.256206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f114ee-50e4-4dd0-9ba2-b6f0c074db61 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:57.256483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f114ee-50e4-4dd0-9ba2-b6f0c074db61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:57.257840 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f114ee-50e4-4dd0-9ba2-b6f0c074db61 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:57.263265 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 341/688] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:12:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:57.266140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a451524-bb37-4311-93c8-fe7a4df3044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:57.266814 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a451524-bb37-4311-93c8-fe7a4df3044e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:57.267093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a451524-bb37-4311-93c8-fe7a4df3044e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:57.267365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a451524-bb37-4311-93c8-fe7a4df3044e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:57.270592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f114ee-50e4-4dd0-9ba2-b6f0c074db61 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:57.278599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a451524-bb37-4311-93c8-fe7a4df3044e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:57.280051 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a451524-bb37-4311-93c8-fe7a4df3044e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:57.284449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-6a451524-bb37-4311-93c8-fe7a4df3044e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:57.295097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:57.306337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:57.307211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:57.307211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:57.307211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:57.307211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 348/689] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:57.307211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a451524-bb37-4311-93c8-fe7a4df3044e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:57.410790 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a451524-bb37-4311-93c8-fe7a4df3044e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:57.415223 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f114ee-50e4-4dd0-9ba2-b6f0c074db61 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:57.449986 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73364390-12e6-4619-9e36-39f0740ec2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:57.450640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364390-12e6-4619-9e36-39f0740ec2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:57.450906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7541210-65d9-429a-bf83-96b27d382e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:57.451029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364390-12e6-4619-9e36-39f0740ec2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:57.451375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364390-12e6-4619-9e36-39f0740ec2f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:57.451491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541210-65d9-429a-bf83-96b27d382e47 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:57.451770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541210-65d9-429a-bf83-96b27d382e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:57.452062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7541210-65d9-429a-bf83-96b27d382e47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:57.452300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73364390-12e6-4619-9e36-39f0740ec2f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:57.452663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73364390-12e6-4619-9e36-39f0740ec2f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:57.452785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7541210-65d9-429a-bf83-96b27d382e47 None admin] RBAC: Adding query filter params (interface=public, service_id=e68dc8ede4df4c3c910faa97fd11388e, region_id=RegionOne) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:57.453093 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7541210-65d9-429a-bf83-96b27d382e47 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:57.454248 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73364390-12e6-4619-9e36-39f0740ec2f0 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:57.454476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7541210-65d9-429a-bf83-96b27d382e47 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:57.460620 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 342/690] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 19 02:12:57 2022] GET /identity/v3/endpoints?service_id=e68dc8ede4df4c3c910faa97fd11388e&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:57.462308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ecbb6-9a0d-4cdc-a61c-08ca3dd8ce75 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:57.462590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ecbb6-9a0d-4cdc-a61c-08ca3dd8ce75 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:57.462859 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ecbb6-9a0d-4cdc-a61c-08ca3dd8ce75 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:57.465260 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 343/691] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:57.470328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ecbb6-9a0d-4cdc-a61c-08ca3dd8ce75 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:57.472636 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:57.472882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 349/692] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:57.474359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94d3a3-66a0-4524-b38e-8d5e09d9e766 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:57.474861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94d3a3-66a0-4524-b38e-8d5e09d9e766 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:57.475329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94d3a3-66a0-4524-b38e-8d5e09d9e766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:57.493326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-dd94d3a3-66a0-4524-b38e-8d5e09d9e766 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:57.494811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-dd94d3a3-66a0-4524-b38e-8d5e09d9e766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:57.544833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:57.567691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 350/693] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:12:57.571057 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94d3a3-66a0-4524-b38e-8d5e09d9e766 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:57.585295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ecbb6-9a0d-4cdc-a61c-08ca3dd8ce75 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:57.631629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e50ab0-f4db-40ea-8235-0b69bbe889fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:57.632889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e50ab0-f4db-40ea-8235-0b69bbe889fc None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:57.633669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e50ab0-f4db-40ea-8235-0b69bbe889fc None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:57.634568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e50ab0-f4db-40ea-8235-0b69bbe889fc None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:57.635531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e50ab0-f4db-40ea-8235-0b69bbe889fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:57.637469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e50ab0-f4db-40ea-8235-0b69bbe889fc None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:57.650990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 344/694] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 19 02:12:57 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:12:57.687006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94d3a3-66a0-4524-b38e-8d5e09d9e766 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:57.718677 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:57.728015 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ee821-e174-45ba-a87f-35db33b1097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:57.728665 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ee821-e174-45ba-a87f-35db33b1097b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:57.728946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ee821-e174-45ba-a87f-35db33b1097b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:57.729217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ee821-e174-45ba-a87f-35db33b1097b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:57.737749 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ee821-e174-45ba-a87f-35db33b1097b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:57.740530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ee821-e174-45ba-a87f-35db33b1097b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-8d5ee821-e174-45ba-a87f-35db33b1097b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:12:57.753095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:57.754151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:57.755130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:57.755130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:57.755130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:12:57.755130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:12:57.755130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:12:57.755130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:12:57.755130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:12:57.755130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:12:57.755130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 19 02:12:57.755130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:57.755130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 351/695] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 19 02:12:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:57.763516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e50ab0-f4db-40ea-8235-0b69bbe889fc None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:57.837845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ee821-e174-45ba-a87f-35db33b1097b None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:57.885938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e50ab0-f4db-40ea-8235-0b69bbe889fc None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:57.929600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d6920-fe40-4b17-b3dc-561908b6397a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:57.930279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d6920-fe40-4b17-b3dc-561908b6397a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:57.930554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d6920-fe40-4b17-b3dc-561908b6397a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:57.930821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d6920-fe40-4b17-b3dc-561908b6397a None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:57.931547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d6920-fe40-4b17-b3dc-561908b6397a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:57.931845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d6920-fe40-4b17-b3dc-561908b6397a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:57.933348 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d6920-fe40-4b17-b3dc-561908b6397a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:57.937007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:57.953123 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 345/696] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Oct 19 02:12:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:57.960296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d6920-fe40-4b17-b3dc-561908b6397a None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:57.968601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ee821-e174-45ba-a87f-35db33b1097b None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:58.010344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33200806-76b2-4e21-ac8c-2f8e0ecf1aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:58.010963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33200806-76b2-4e21-ac8c-2f8e0ecf1aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:58.011238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33200806-76b2-4e21-ac8c-2f8e0ecf1aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:58.011502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33200806-76b2-4e21-ac8c-2f8e0ecf1aa3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:58.012101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33200806-76b2-4e21-ac8c-2f8e0ecf1aa3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:58.014126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33200806-76b2-4e21-ac8c-2f8e0ecf1aa3 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-33200806-76b2-4e21-ac8c-2f8e0ecf1aa3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:58.025454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:58.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:58.027352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 352/697] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:58.032331 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33200806-76b2-4e21-ac8c-2f8e0ecf1aa3 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:58.082358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d6920-fe40-4b17-b3dc-561908b6397a None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:58.094291 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:58.119173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10ee33-e914-4641-a142-04fe533b71c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:58.120005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10ee33-e914-4641-a142-04fe533b71c3 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:58.120456 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10ee33-e914-4641-a142-04fe533b71c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:58.120915 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10ee33-e914-4641-a142-04fe533b71c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:58.153141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33200806-76b2-4e21-ac8c-2f8e0ecf1aa3 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:58.182380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:58.190827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10ee33-e914-4641-a142-04fe533b71c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:58.190959 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a47b3-bf7e-4b9a-ae9c-37c6872426e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:58.191452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a47b3-bf7e-4b9a-ae9c-37c6872426e0 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:58.191736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a47b3-bf7e-4b9a-ae9c-37c6872426e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:58.207279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a47b3-bf7e-4b9a-ae9c-37c6872426e0 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:58.207279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a47b3-bf7e-4b9a-ae9c-37c6872426e0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:58.207279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a47b3-bf7e-4b9a-ae9c-37c6872426e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:58.207954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10ee33-e914-4641-a142-04fe533b71c3 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:58.208060 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a47b3-bf7e-4b9a-ae9c-37c6872426e0 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:58.208060 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 353/698] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:12:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:58.208060 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a47b3-bf7e-4b9a-ae9c-37c6872426e0 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:58.324578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a47b3-bf7e-4b9a-ae9c-37c6872426e0 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:58.334203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 346/699] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:12:57 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 02:12:58.336286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10ee33-e914-4641-a142-04fe533b71c3 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:58.350712 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:58.375825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb491c-f1ac-4a8b-95ee-ef0283cc9efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:58.377287 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb491c-f1ac-4a8b-95ee-ef0283cc9efb None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:58.377772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb491c-f1ac-4a8b-95ee-ef0283cc9efb None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:58.378272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb491c-f1ac-4a8b-95ee-ef0283cc9efb None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:58.379586 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb491c-f1ac-4a8b-95ee-ef0283cc9efb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:58.380094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb491c-f1ac-4a8b-95ee-ef0283cc9efb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:58.382368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb491c-f1ac-4a8b-95ee-ef0283cc9efb None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:58.389571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 354/700] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:12:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:58.443542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:58.451214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10ee33-e914-4641-a142-04fe533b71c3 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:58.491357 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54a9a2-d70f-4a71-912b-add251524718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:58.492061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54a9a2-d70f-4a71-912b-add251524718 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:58.492331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54a9a2-d70f-4a71-912b-add251524718 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:58.492598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54a9a2-d70f-4a71-912b-add251524718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:58.504592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54a9a2-d70f-4a71-912b-add251524718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:58.506068 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54a9a2-d70f-4a71-912b-add251524718 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-5d54a9a2-d70f-4a71-912b-add251524718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:12:58.522243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:58.523235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:12:58.524111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:12:58.524111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:12:58.524111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:58.524111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 347/701] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:12:58.530882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb491c-f1ac-4a8b-95ee-ef0283cc9efb None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:58.643022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb491c-f1ac-4a8b-95ee-ef0283cc9efb None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:58.678468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f7e53-dc22-4f8d-9624-6c5f3a659955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:58.679426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f7e53-dc22-4f8d-9624-6c5f3a659955 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:58.679910 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f7e53-dc22-4f8d-9624-6c5f3a659955 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:58.680496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f7e53-dc22-4f8d-9624-6c5f3a659955 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:58.681335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f7e53-dc22-4f8d-9624-6c5f3a659955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:58.681871 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f7e53-dc22-4f8d-9624-6c5f3a659955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:58.683456 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f7e53-dc22-4f8d-9624-6c5f3a659955 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:58.696245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 355/702] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:12:58.702611 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54a9a2-d70f-4a71-912b-add251524718 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:58.807562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54a9a2-d70f-4a71-912b-add251524718 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:58.841949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61552a-ba20-440d-b71e-a065a6e9c17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:58.842685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61552a-ba20-440d-b71e-a065a6e9c17e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:58.843012 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61552a-ba20-440d-b71e-a065a6e9c17e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:58.843283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61552a-ba20-440d-b71e-a065a6e9c17e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:58.849850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61552a-ba20-440d-b71e-a065a6e9c17e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:58.851301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61552a-ba20-440d-b71e-a065a6e9c17e None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-cb61552a-ba20-440d-b71e-a065a6e9c17e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:12:58.862893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:58.863812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:58.864739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:12:58.864739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:12:58.864739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:12:58.864739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:12:58.864739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:12:58.864739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:12:58.864739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:12:58.864739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:12:58.864739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 348/703] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:12:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:58.871571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f7e53-dc22-4f8d-9624-6c5f3a659955 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:58.916548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:58.979089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f7e53-dc22-4f8d-9624-6c5f3a659955 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:59.020612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacbda3-70c5-468d-8bd2-b92c67553463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:59.021648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacbda3-70c5-468d-8bd2-b92c67553463 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:59.022179 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacbda3-70c5-468d-8bd2-b92c67553463 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:59.022678 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacbda3-70c5-468d-8bd2-b92c67553463 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:59.023566 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacbda3-70c5-468d-8bd2-b92c67553463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:59.024071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacbda3-70c5-468d-8bd2-b92c67553463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:59.025693 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacbda3-70c5-468d-8bd2-b92c67553463 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:59.040205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 356/704] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:12:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:59.046564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61552a-ba20-440d-b71e-a065a6e9c17e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:59.058418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:59.100041 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba88bc-8d83-42aa-8e5f-1f2fb5721dbe None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:59.100371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba88bc-8d83-42aa-8e5f-1f2fb5721dbe None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:59.102013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba88bc-8d83-42aa-8e5f-1f2fb5721dbe None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:59.102013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 357/705] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:12:59.111327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad104b1-cca6-4989-8502-78634de1e3cd None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:59.111612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad104b1-cca6-4989-8502-78634de1e3cd None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:59.111885 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad104b1-cca6-4989-8502-78634de1e3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:59.131346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-0ad104b1-cca6-4989-8502-78634de1e3cd None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:59.132522 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0ad104b1-cca6-4989-8502-78634de1e3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:12:59.163184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61552a-ba20-440d-b71e-a065a6e9c17e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:59.197562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89340a9e-e6d5-492e-ad6c-9213e39a447e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:59.198531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89340a9e-e6d5-492e-ad6c-9213e39a447e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:59.199008 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89340a9e-e6d5-492e-ad6c-9213e39a447e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:59.199503 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89340a9e-e6d5-492e-ad6c-9213e39a447e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:59.200419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89340a9e-e6d5-492e-ad6c-9213e39a447e None admin] RBAC: Adding query filter params (role.id=e95a043c68e440b2a961391d6e6e625b, scope.project.id=676e986ff972410d9f498800d313476f, user.id=717c74cb769148a5b465079add961892) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:59.200919 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89340a9e-e6d5-492e-ad6c-9213e39a447e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:59.202607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89340a9e-e6d5-492e-ad6c-9213e39a447e None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:59.214755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 358/706] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:12:59.219261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 349/707] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:12:59 2022] GET /identity/v3/role_assignments?user.id=717c74cb769148a5b465079add961892&scope.project.id=676e986ff972410d9f498800d313476f&role.id=e95a043c68e440b2a961391d6e6e625b => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:59.471564 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad104b1-cca6-4989-8502-78634de1e3cd None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:59.502391 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:59.578180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad104b1-cca6-4989-8502-78634de1e3cd None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:59.614416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116c3cf-5be5-4ba0-bc3b-3d58dbd1907a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:59.615422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116c3cf-5be5-4ba0-bc3b-3d58dbd1907a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:59.615909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116c3cf-5be5-4ba0-bc3b-3d58dbd1907a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:59.616380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116c3cf-5be5-4ba0-bc3b-3d58dbd1907a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:59.617170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116c3cf-5be5-4ba0-bc3b-3d58dbd1907a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:59.618766 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116c3cf-5be5-4ba0-bc3b-3d58dbd1907a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-f116c3cf-5be5-4ba0-bc3b-3d58dbd1907a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:12:59.628143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:12:59.629115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:12:59.629939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:12:59.632704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 359/708] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 19 02:12:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:12:59.638310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89340a9e-e6d5-492e-ad6c-9213e39a447e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:59.714711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:59.747191 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89340a9e-e6d5-492e-ad6c-9213e39a447e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:59.788438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cc4dc-6456-48d2-b735-8bbb48a08214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:59.789316 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cc4dc-6456-48d2-b735-8bbb48a08214 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:59.789749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cc4dc-6456-48d2-b735-8bbb48a08214 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:59.790200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cc4dc-6456-48d2-b735-8bbb48a08214 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:59.790964 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cc4dc-6456-48d2-b735-8bbb48a08214 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:59.791424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cc4dc-6456-48d2-b735-8bbb48a08214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:59.792904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07cc4dc-6456-48d2-b735-8bbb48a08214 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:59.798698 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 350/709] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:12:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:59.806433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116c3cf-5be5-4ba0-bc3b-3d58dbd1907a None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:59.905733 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:12:59.912353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116c3cf-5be5-4ba0-bc3b-3d58dbd1907a None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:12:59.947309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021f5b2-bd8a-4da0-a2a3-e6a532537949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:12:59.947913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021f5b2-bd8a-4da0-a2a3-e6a532537949 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:59.948189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021f5b2-bd8a-4da0-a2a3-e6a532537949 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:59.948458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021f5b2-bd8a-4da0-a2a3-e6a532537949 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:59.949078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021f5b2-bd8a-4da0-a2a3-e6a532537949 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:12:59.949376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021f5b2-bd8a-4da0-a2a3-e6a532537949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:12:59.950842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021f5b2-bd8a-4da0-a2a3-e6a532537949 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:12:59.956669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 360/710] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:12:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:12:59.962227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cc4dc-6456-48d2-b735-8bbb48a08214 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:12:59.969853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84667846-c4e7-493e-909d-6fc651229184 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:59.970188 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84667846-c4e7-493e-909d-6fc651229184 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:59.970468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84667846-c4e7-493e-909d-6fc651229184 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:59.973059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 361/711] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:12:59.976953 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528039b7-f4f0-4836-95c7-d6cad8452f6e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:12:59.977240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528039b7-f4f0-4836-95c7-d6cad8452f6e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:12:59.977514 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528039b7-f4f0-4836-95c7-d6cad8452f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:12:59.995461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-528039b7-f4f0-4836-95c7-d6cad8452f6e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:12:59.996649 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-528039b7-f4f0-4836-95c7-d6cad8452f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:00.076201 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 362/712] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:12:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:13:00.080037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07cc4dc-6456-48d2-b735-8bbb48a08214 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:00.125781 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5034e2-3727-4996-869c-a14ea1b5d280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:00.126508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5034e2-3727-4996-869c-a14ea1b5d280 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:00.126798 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5034e2-3727-4996-869c-a14ea1b5d280 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:00.127072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5034e2-3727-4996-869c-a14ea1b5d280 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:00.127767 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5034e2-3727-4996-869c-a14ea1b5d280 None admin] RBAC: Adding query filter params (interface=public, service_id=4023d4c1025e4a47bd72e186f27b543c, region_id=RegionOne) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:00.128072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5034e2-3727-4996-869c-a14ea1b5d280 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:00.129490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5034e2-3727-4996-869c-a14ea1b5d280 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:00.134907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 351/713] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 19 02:12:59 2022] GET /identity/v3/endpoints?service_id=4023d4c1025e4a47bd72e186f27b543c&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:00.135382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:00.141141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528039b7-f4f0-4836-95c7-d6cad8452f6e None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:00.240150 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:00.258110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528039b7-f4f0-4836-95c7-d6cad8452f6e None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:00.292944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd66b82-9ea2-4729-8631-46a9b74d65b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:00.293779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd66b82-9ea2-4729-8631-46a9b74d65b8 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:00.294229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd66b82-9ea2-4729-8631-46a9b74d65b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:00.294671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd66b82-9ea2-4729-8631-46a9b74d65b8 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:00.295408 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd66b82-9ea2-4729-8631-46a9b74d65b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:00.297590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd66b82-9ea2-4729-8631-46a9b74d65b8 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:00.304231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 363/714] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 19 02:13:00 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:13:00.382799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5034e2-3727-4996-869c-a14ea1b5d280 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:00.496996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5034e2-3727-4996-869c-a14ea1b5d280 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:00.533295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e5a4d-aec1-4a4a-90e0-d5098d365c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:00.534250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e5a4d-aec1-4a4a-90e0-d5098d365c20 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:00.534696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e5a4d-aec1-4a4a-90e0-d5098d365c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:00.535116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e5a4d-aec1-4a4a-90e0-d5098d365c20 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:00.535985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e5a4d-aec1-4a4a-90e0-d5098d365c20 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:00.537581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e5a4d-aec1-4a4a-90e0-d5098d365c20 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-ca4e5a4d-aec1-4a4a-90e0-d5098d365c20 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:00.547874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:00.549149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 19 02:13:00.550245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:00.552876 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 352/715] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:13:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:00.558934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd66b82-9ea2-4729-8631-46a9b74d65b8 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:00.656452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e270be-c299-4a22-abe7-1b2e2b6c493b None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:00.656952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e270be-c299-4a22-abe7-1b2e2b6c493b None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:00.656952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e270be-c299-4a22-abe7-1b2e2b6c493b None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:00.658525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 353/716] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:13:00.663557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c064a9e-1072-4cc8-9bae-52f9e3bae12a None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:00.663839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c064a9e-1072-4cc8-9bae-52f9e3bae12a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:00.664112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c064a9e-1072-4cc8-9bae-52f9e3bae12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:00.671384 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd66b82-9ea2-4729-8631-46a9b74d65b8 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:00.682893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7c064a9e-1072-4cc8-9bae-52f9e3bae12a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:00.684061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7c064a9e-1072-4cc8-9bae-52f9e3bae12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:00.710835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536dc72-dacf-49eb-afab-c513d140fb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:00.711439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536dc72-dacf-49eb-afab-c513d140fb2f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:00.711810 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536dc72-dacf-49eb-afab-c513d140fb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:00.712063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e536dc72-dacf-49eb-afab-c513d140fb2f None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:00.712682 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536dc72-dacf-49eb-afab-c513d140fb2f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:00.712986 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536dc72-dacf-49eb-afab-c513d140fb2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:00.714364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e536dc72-dacf-49eb-afab-c513d140fb2f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:00.720210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 364/717] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:13:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:00.727487 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536dc72-dacf-49eb-afab-c513d140fb2f None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:00.746965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:00.762442 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:00.771457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 354/718] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:13:00.840282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e536dc72-dacf-49eb-afab-c513d140fb2f None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:00.875195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8ce7f-dcdb-44de-8cbe-ae956f6fcfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:00.876090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8ce7f-dcdb-44de-8cbe-ae956f6fcfb6 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:00.876548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8ce7f-dcdb-44de-8cbe-ae956f6fcfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:00.876998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8ce7f-dcdb-44de-8cbe-ae956f6fcfb6 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:00.877801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8ce7f-dcdb-44de-8cbe-ae956f6fcfb6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:00.878330 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8ce7f-dcdb-44de-8cbe-ae956f6fcfb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:00.879884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8ce7f-dcdb-44de-8cbe-ae956f6fcfb6 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:00.890217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 365/719] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Oct 19 02:13:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:00.891935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c064a9e-1072-4cc8-9bae-52f9e3bae12a None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:00.992024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c064a9e-1072-4cc8-9bae-52f9e3bae12a None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:01.027915 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3fc37-54bb-4dec-9859-fa0f97bf0f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:01.028529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3fc37-54bb-4dec-9859-fa0f97bf0f27 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:01.028804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3fc37-54bb-4dec-9859-fa0f97bf0f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:01.029074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3fc37-54bb-4dec-9859-fa0f97bf0f27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:01.029816 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3fc37-54bb-4dec-9859-fa0f97bf0f27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:01.031399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3fc37-54bb-4dec-9859-fa0f97bf0f27 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:01.075178 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8ce7f-dcdb-44de-8cbe-ae956f6fcfb6 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:01.112404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 355/720] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:13:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:13:01.180182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:01.201082 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8ce7f-dcdb-44de-8cbe-ae956f6fcfb6 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:01.246414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a91d56-aa42-4d7d-a583-9708b1d4642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:01.247027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a91d56-aa42-4d7d-a583-9708b1d4642e None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:01.247297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a91d56-aa42-4d7d-a583-9708b1d4642e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:01.247565 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a91d56-aa42-4d7d-a583-9708b1d4642e None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:01.248222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a91d56-aa42-4d7d-a583-9708b1d4642e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:01.249650 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a91d56-aa42-4d7d-a583-9708b1d4642e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:01.320598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 366/721] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:13:01 2022] POST /identity/v3/services => generated 242 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:13:01.500277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a741f-6a0b-4bcd-9b40-9909433c622d None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:01.500620 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a741f-6a0b-4bcd-9b40-9909433c622d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:01.500792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a741f-6a0b-4bcd-9b40-9909433c622d None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:01.502350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 356/722] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:01.507530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d889b18-1c58-49b1-abf1-37f12a903a4e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:01.508068 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d889b18-1c58-49b1-abf1-37f12a903a4e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:01.508539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d889b18-1c58-49b1-abf1-37f12a903a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:01.529671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-4d889b18-1c58-49b1-abf1-37f12a903a4e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:01.531671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4d889b18-1c58-49b1-abf1-37f12a903a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:01.607061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 367/723] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:01.876669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a741f-6a0b-4bcd-9b40-9909433c622d None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:01.926319 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:01.978901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a741f-6a0b-4bcd-9b40-9909433c622d None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:02.013287 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7b6c2-c12d-4c47-a872-ca1e1731976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:02.013915 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7b6c2-c12d-4c47-a872-ca1e1731976b None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:02.014218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7b6c2-c12d-4c47-a872-ca1e1731976b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:02.014484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7b6c2-c12d-4c47-a872-ca1e1731976b None admin] PATH_INFO: `/v3/services/image` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:02.015649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7b6c2-c12d-4c47-a872-ca1e1731976b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:02.017000 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7b6c2-c12d-4c47-a872-ca1e1731976b None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-c0b7b6c2-c12d-4c47-a872-ca1e1731976b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:02.025801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:02.026799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:02.027633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 357/724] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Oct 19 02:13:01 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:02.035105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d889b18-1c58-49b1-abf1-37f12a903a4e None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:02.104761 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:02.146669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d889b18-1c58-49b1-abf1-37f12a903a4e None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:02.182816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a3c43-cfe2-4172-aee1-0fa52f8e9989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:02.184304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a3c43-cfe2-4172-aee1-0fa52f8e9989 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:02.184304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a3c43-cfe2-4172-aee1-0fa52f8e9989 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:02.184304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a3c43-cfe2-4172-aee1-0fa52f8e9989 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:02.184583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a3c43-cfe2-4172-aee1-0fa52f8e9989 None admin] RBAC: Adding query filter params (name=image) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:02.184842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a3c43-cfe2-4172-aee1-0fa52f8e9989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:02.186234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a3c43-cfe2-4172-aee1-0fa52f8e9989 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:02.191901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 368/725] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:02.197543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7b6c2-c12d-4c47-a872-ca1e1731976b None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:02.306557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7b6c2-c12d-4c47-a872-ca1e1731976b None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:02.320054 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:02.341666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55e566-e160-4cb1-9e25-c11af0f489c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:02.342300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55e566-e160-4cb1-9e25-c11af0f489c0 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:02.342574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55e566-e160-4cb1-9e25-c11af0f489c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:02.342923 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55e566-e160-4cb1-9e25-c11af0f489c0 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:02.343548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55e566-e160-4cb1-9e25-c11af0f489c0 None admin] RBAC: Adding query filter params (type=image) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:02.343842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55e566-e160-4cb1-9e25-c11af0f489c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:02.345187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55e566-e160-4cb1-9e25-c11af0f489c0 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:02.349963 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 358/726] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:02.611247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417dad-c57e-4144-9a89-cdda60020be9 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:02.612264 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417dad-c57e-4144-9a89-cdda60020be9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:02.612754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9417dad-c57e-4144-9a89-cdda60020be9 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:02.614625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 369/727] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:02.620249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829bbd2-90f0-486d-b936-83702ec46195 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:02.620774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829bbd2-90f0-486d-b936-83702ec46195 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:02.621253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829bbd2-90f0-486d-b936-83702ec46195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:02.640595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-a829bbd2-90f0-486d-b936-83702ec46195 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:02.642139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a829bbd2-90f0-486d-b936-83702ec46195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:02.710332 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 359/728] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:02.980762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9417dad-c57e-4144-9a89-cdda60020be9 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:03.085837 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9417dad-c57e-4144-9a89-cdda60020be9 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:03.118017 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:03.126932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38d6e3-0d6b-437a-b441-af479c9a5736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:03.127562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38d6e3-0d6b-437a-b441-af479c9a5736 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:03.127838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38d6e3-0d6b-437a-b441-af479c9a5736 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:03.128106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38d6e3-0d6b-437a-b441-af479c9a5736 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:03.128683 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38d6e3-0d6b-437a-b441-af479c9a5736 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38d6e3-0d6b-437a-b441-af479c9a5736 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-9a38d6e3-0d6b-437a-b441-af479c9a5736 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:03.141217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:03.142123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:13:03.143013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:03.143844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 370/729] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 19 02:13:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:03.150253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829bbd2-90f0-486d-b936-83702ec46195 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:03.257064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829bbd2-90f0-486d-b936-83702ec46195 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:03.266968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:03.297350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ca658-aae0-47c2-a142-bafe651060aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:03.298273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ca658-aae0-47c2-a142-bafe651060aa None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:03.298692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ca658-aae0-47c2-a142-bafe651060aa None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:03.299100 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ca658-aae0-47c2-a142-bafe651060aa None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:03.299984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ca658-aae0-47c2-a142-bafe651060aa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:03.300425 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ca658-aae0-47c2-a142-bafe651060aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:03.301910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ca658-aae0-47c2-a142-bafe651060aa None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:03.308266 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 360/730] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:13:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:03.314278 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38d6e3-0d6b-437a-b441-af479c9a5736 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:03.426464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38d6e3-0d6b-437a-b441-af479c9a5736 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:03.432498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f14884-6868-42d8-892d-59e9d5802a54 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:03.432773 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f14884-6868-42d8-892d-59e9d5802a54 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:03.433281 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f14884-6868-42d8-892d-59e9d5802a54 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:03.435114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 361/731] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:13:03.440951 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672b19c-0f8f-428f-b7b9-5d0c6833ab3f None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:03.441457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672b19c-0f8f-428f-b7b9-5d0c6833ab3f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:03.441924 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672b19c-0f8f-428f-b7b9-5d0c6833ab3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:03.460275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-d672b19c-0f8f-428f-b7b9-5d0c6833ab3f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:03.461772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d672b19c-0f8f-428f-b7b9-5d0c6833ab3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:03.464158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf22c1b-7de0-440a-b585-93f7e9bb7314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:03.465088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf22c1b-7de0-440a-b585-93f7e9bb7314 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:03.466110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf22c1b-7de0-440a-b585-93f7e9bb7314 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:03.471024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf22c1b-7de0-440a-b585-93f7e9bb7314 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:03.471873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf22c1b-7de0-440a-b585-93f7e9bb7314 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:03.472174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf22c1b-7de0-440a-b585-93f7e9bb7314 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:03.473535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf22c1b-7de0-440a-b585-93f7e9bb7314 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:03.478680 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 371/732] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:13:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:03.484062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf22c1b-7de0-440a-b585-93f7e9bb7314 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:03.545125 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:03.554357 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 362/733] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:13:03.595685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf22c1b-7de0-440a-b585-93f7e9bb7314 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:03.630229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebd0df-1236-4e9e-8047-bffaea7b09c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:03.631551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebd0df-1236-4e9e-8047-bffaea7b09c1 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:03.631973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebd0df-1236-4e9e-8047-bffaea7b09c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:03.632388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebd0df-1236-4e9e-8047-bffaea7b09c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:03.633197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aebd0df-1236-4e9e-8047-bffaea7b09c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:03.634720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aebd0df-1236-4e9e-8047-bffaea7b09c1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:03.648798 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ee510-a4b1-43b1-a8fc-f3b0e44a9ec3 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:03.649233 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ee510-a4b1-43b1-a8fc-f3b0e44a9ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:03.649573 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ee510-a4b1-43b1-a8fc-f3b0e44a9ec3 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:03.651179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 363/734] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:13:03.656531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d49753-b642-48d7-9324-26002e483f08 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:03.656833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d49753-b642-48d7-9324-26002e483f08 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:03.657078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d49753-b642-48d7-9324-26002e483f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:03.675326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-32d49753-b642-48d7-9324-26002e483f08 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:03.676514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-32d49753-b642-48d7-9324-26002e483f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:03.752806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 364/736] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:13:03.753390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 372/736] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:13:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:13:03.823920 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d49753-b642-48d7-9324-26002e483f08 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:03.856848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:03.930586 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d49753-b642-48d7-9324-26002e483f08 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:03.973229 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ec692-8f03-48dd-8bfa-3c4ab4ea6b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:03.973837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ec692-8f03-48dd-8bfa-3c4ab4ea6b01 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:03.974128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ec692-8f03-48dd-8bfa-3c4ab4ea6b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:03.974398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ec692-8f03-48dd-8bfa-3c4ab4ea6b01 None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:03.975030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ec692-8f03-48dd-8bfa-3c4ab4ea6b01 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:03.976424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ec692-8f03-48dd-8bfa-3c4ab4ea6b01 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:03.979004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: WARNING keystone.server.flask.common [None req-a72ec692-8f03-48dd-8bfa-3c4ab4ea6b01 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 19 02:13:04.016521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebd0df-1236-4e9e-8047-bffaea7b09c1 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:04.025392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.sql.core [None req-a72ec692-8f03-48dd-8bfa-3c4ab4ea6b01 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 19 02:13:04.025392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 02:13:04.025392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [parameters: {'id': 'f0146672c27d47df9d1e2d7b9115da83', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 19 02:13:04.025392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81616) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-a72ec692-8f03-48dd-8bfa-3c4ab4ea6b01 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f0146672c27d47df9d1e2d7b9115da83.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f0146672c27d47df9d1e2d7b9115da83. Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application conn.query(q) Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 02:13:04.033499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result.read() Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 19 02:13:04.034383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application next(self.gen) Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application yield resource Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application next(self.gen) Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.session.rollback() Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.force_reraise() Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise self.value Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application session.commit() Oct 19 02:13:04.035780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._prepare_impl() Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.session.flush() Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._flush(objects) Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application compat.raise_( Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application flush_context.execute() Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 19 02:13:04.036925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rec.execute(self) Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 19 02:13:04.037869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application conn.query(q) Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result.read() Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 02:13:04.040612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [parameters: {'id': 'f0146672c27d47df9d1e2d7b9115da83', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:04.042827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:04.044832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:04.047283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:04.047283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 19 02:13:04.047283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 19 02:13:04.047283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:04.047283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:04.047283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 19 02:13:04.047283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 19 02:13:04.047283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f0146672c27d47df9d1e2d7b9115da83. Oct 19 02:13:04.047283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:04.047283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 365/737] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Oct 19 02:13:03 2022] POST /identity/v3/projects => generated 262 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:04.047283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ec692-8f03-48dd-8bfa-3c4ab4ea6b01 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:04.136163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:04.143205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebd0df-1236-4e9e-8047-bffaea7b09c1 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:04.154584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:04.156631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ec692-8f03-48dd-8bfa-3c4ab4ea6b01 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:04.178628 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4bccf-153f-4d26-8282-500d439b4d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:04.179781 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4bccf-153f-4d26-8282-500d439b4d00 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:04.180062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4bccf-153f-4d26-8282-500d439b4d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:04.180329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4bccf-153f-4d26-8282-500d439b4d00 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:04.180909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4bccf-153f-4d26-8282-500d439b4d00 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:04.182440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4bccf-153f-4d26-8282-500d439b4d00 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-bef4bccf-153f-4d26-8282-500d439b4d00 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:04.192235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:04.193461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:04.194494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 373/738] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:13:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:04.197887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a46e70-7eba-4619-a9d9-fe6477b844c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:04.198610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46e70-7eba-4619-a9d9-fe6477b844c8 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:04.200717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46e70-7eba-4619-a9d9-fe6477b844c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:04.200717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a46e70-7eba-4619-a9d9-fe6477b844c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:04.200895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4bccf-153f-4d26-8282-500d439b4d00 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:04.206610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a46e70-7eba-4619-a9d9-fe6477b844c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:04.208304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a46e70-7eba-4619-a9d9-fe6477b844c8 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-14a46e70-7eba-4619-a9d9-fe6477b844c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:13:04.220780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:04.221721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:04.222662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:13:04.222662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:13:04.222662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:13:04.222662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:13:04.222662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:13:04.222662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:13:04.222662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:13:04.222662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:04.226363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 366/739] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:13:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:04.227959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a46e70-7eba-4619-a9d9-fe6477b844c8 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:04.315579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4bccf-153f-4d26-8282-500d439b4d00 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:04.335897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a46e70-7eba-4619-a9d9-fe6477b844c8 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:04.355758 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ded9d-7a83-41b9-a404-42b6987b34e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:04.356486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ded9d-7a83-41b9-a404-42b6987b34e9 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:04.356886 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ded9d-7a83-41b9-a404-42b6987b34e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:04.357267 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ded9d-7a83-41b9-a404-42b6987b34e9 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:04.358130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ded9d-7a83-41b9-a404-42b6987b34e9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:04.359120 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ded9d-7a83-41b9-a404-42b6987b34e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:04.360587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ded9d-7a83-41b9-a404-42b6987b34e9 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:04.370126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 374/740] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:13:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:04.371876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ded9d-7a83-41b9-a404-42b6987b34e9 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:04.375552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ba3d2-fb51-42a1-8efb-d209fe090426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:04.376361 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ba3d2-fb51-42a1-8efb-d209fe090426 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:04.376747 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ba3d2-fb51-42a1-8efb-d209fe090426 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:04.377849 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ba3d2-fb51-42a1-8efb-d209fe090426 None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:04.378012 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ba3d2-fb51-42a1-8efb-d209fe090426 None admin] RBAC: Adding query filter params (name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:04.378429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ba3d2-fb51-42a1-8efb-d209fe090426 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:04.379938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ba3d2-fb51-42a1-8efb-d209fe090426 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:04.406698 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 367/741] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:13:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:04.417555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:04.493037 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ded9d-7a83-41b9-a404-42b6987b34e9 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:04.532860 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8539c36-0958-4e05-b423-ec1539624f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:04.533471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8539c36-0958-4e05-b423-ec1539624f42 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:04.533743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8539c36-0958-4e05-b423-ec1539624f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:04.534009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8539c36-0958-4e05-b423-ec1539624f42 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:04.534676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8539c36-0958-4e05-b423-ec1539624f42 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:04.534971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8539c36-0958-4e05-b423-ec1539624f42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:04.536338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8539c36-0958-4e05-b423-ec1539624f42 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:04.542272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 375/742] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:13:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:04.548254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ba3d2-fb51-42a1-8efb-d209fe090426 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:04.556948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:04.601055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7271-56a9-42a0-83dc-9d0447dda1bc None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:04.601055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7271-56a9-42a0-83dc-9d0447dda1bc None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:04.601327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7271-56a9-42a0-83dc-9d0447dda1bc None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:04.603648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 376/743] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:13:04.610474 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab49bd7-2689-4885-8f0e-d0bcc09a591a None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:04.610863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab49bd7-2689-4885-8f0e-d0bcc09a591a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:04.611233 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab49bd7-2689-4885-8f0e-d0bcc09a591a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:04.630578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-1ab49bd7-2689-4885-8f0e-d0bcc09a591a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:04.631901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1ab49bd7-2689-4885-8f0e-d0bcc09a591a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:04.675060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ba3d2-fb51-42a1-8efb-d209fe090426 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:04.712259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a628326-b313-45ac-966b-d0a4673943af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:04.712929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628326-b313-45ac-966b-d0a4673943af None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:04.713277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628326-b313-45ac-966b-d0a4673943af None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:04.713480 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628326-b313-45ac-966b-d0a4673943af None admin] PATH_INFO: `/v3/endpoints` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:04.714270 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a628326-b313-45ac-966b-d0a4673943af None admin] RBAC: Adding query filter params (interface=public, service_id=16e996ee14b54f07b09024a02a0630b6, region_id=RegionOne) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:04.714561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a628326-b313-45ac-966b-d0a4673943af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:04.716196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a628326-b313-45ac-966b-d0a4673943af None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:04.720172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 377/744] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:04.721958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 368/745] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 19 02:13:04 2022] GET /identity/v3/endpoints?service_id=16e996ee14b54f07b09024a02a0630b6&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:04.727783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab49bd7-2689-4885-8f0e-d0bcc09a591a None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:04.759946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:04.832780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab49bd7-2689-4885-8f0e-d0bcc09a591a None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:04.869787 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c021e5-7269-4592-bcb8-8bc6167f4dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:04.870735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c021e5-7269-4592-bcb8-8bc6167f4dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:04.871154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c021e5-7269-4592-bcb8-8bc6167f4dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:04.871567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c021e5-7269-4592-bcb8-8bc6167f4dc5 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:04.872300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c021e5-7269-4592-bcb8-8bc6167f4dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:04.873782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c021e5-7269-4592-bcb8-8bc6167f4dc5 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:04.880442 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 378/746] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 19 02:13:04 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:13:04.989373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a628326-b313-45ac-966b-d0a4673943af None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:05.063471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:05.105647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a628326-b313-45ac-966b-d0a4673943af None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:05.145390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ed1f3-5e8d-4125-85ee-6b4357e8893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:05.145923 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ed1f3-5e8d-4125-85ee-6b4357e8893c None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:05.146233 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ed1f3-5e8d-4125-85ee-6b4357e8893c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:05.146505 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ed1f3-5e8d-4125-85ee-6b4357e8893c None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:05.147174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ed1f3-5e8d-4125-85ee-6b4357e8893c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:05.148519 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ed1f3-5e8d-4125-85ee-6b4357e8893c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:05.227523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 369/747] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:13:04 2022] POST /identity/v3/services => generated 242 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:13:06.090001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d0ae3-1b36-4bd3-9ce0-51d394e3608d None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:06.090529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d0ae3-1b36-4bd3-9ce0-51d394e3608d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:06.090606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d0ae3-1b36-4bd3-9ce0-51d394e3608d None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:06.092260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 379/748] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:06.109220 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338ebb3-dbb5-40b5-ad62-dd1f55bed483 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:06.109902 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338ebb3-dbb5-40b5-ad62-dd1f55bed483 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:06.110409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338ebb3-dbb5-40b5-ad62-dd1f55bed483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:06.130811 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-5338ebb3-dbb5-40b5-ad62-dd1f55bed483 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:06.132698 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5338ebb3-dbb5-40b5-ad62-dd1f55bed483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:06.216922 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 370/749] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:06 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:06.490261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d0ae3-1b36-4bd3-9ce0-51d394e3608d None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:06.588571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:06.597271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d0ae3-1b36-4bd3-9ce0-51d394e3608d None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:06.633770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc65387-8c92-49cb-8f35-49a35f51fcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:06.634449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc65387-8c92-49cb-8f35-49a35f51fcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:06.634722 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc65387-8c92-49cb-8f35-49a35f51fcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:06.634991 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc65387-8c92-49cb-8f35-49a35f51fcb1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:06.635662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc65387-8c92-49cb-8f35-49a35f51fcb1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:06.636978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc65387-8c92-49cb-8f35-49a35f51fcb1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-4fc65387-8c92-49cb-8f35-49a35f51fcb1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:06.647077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:06.653114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:06.653952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 380/750] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 19 02:13:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:06.658331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338ebb3-dbb5-40b5-ad62-dd1f55bed483 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:06.714363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:06.741402 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde65aba-9c86-400b-b1c7-e73e7cef6f04 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:06.741600 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde65aba-9c86-400b-b1c7-e73e7cef6f04 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:06.741876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde65aba-9c86-400b-b1c7-e73e7cef6f04 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:06.743354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 381/751] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:06.751469 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797186f8-2563-4bb5-85ff-3e2a00d5465e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:06.751748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797186f8-2563-4bb5-85ff-3e2a00d5465e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:06.752020 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797186f8-2563-4bb5-85ff-3e2a00d5465e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:06.770657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-797186f8-2563-4bb5-85ff-3e2a00d5465e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:06.771824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-797186f8-2563-4bb5-85ff-3e2a00d5465e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:06.773815 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338ebb3-dbb5-40b5-ad62-dd1f55bed483 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:06.818138 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f87d4-5ac4-4deb-a3ba-234ffdacb344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:06.819533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f87d4-5ac4-4deb-a3ba-234ffdacb344 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:06.820089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f87d4-5ac4-4deb-a3ba-234ffdacb344 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:06.820549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f87d4-5ac4-4deb-a3ba-234ffdacb344 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:06.821530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f87d4-5ac4-4deb-a3ba-234ffdacb344 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:06.822003 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f87d4-5ac4-4deb-a3ba-234ffdacb344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:06.823526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f87d4-5ac4-4deb-a3ba-234ffdacb344 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:06.830553 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 371/752] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:13:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:06.835835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f87d4-5ac4-4deb-a3ba-234ffdacb344 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:06.854154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 382/753] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:06 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:06.941054 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f87d4-5ac4-4deb-a3ba-234ffdacb344 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:06.958592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:06.977558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075e9c6-a968-4f3b-9a2c-ed5fa270da3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:06.978397 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e9c6-a968-4f3b-9a2c-ed5fa270da3c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:06.980843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e9c6-a968-4f3b-9a2c-ed5fa270da3c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:06.980843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075e9c6-a968-4f3b-9a2c-ed5fa270da3c None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:06.980843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075e9c6-a968-4f3b-9a2c-ed5fa270da3c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:06.980843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075e9c6-a968-4f3b-9a2c-ed5fa270da3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:06.982169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5075e9c6-a968-4f3b-9a2c-ed5fa270da3c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:06.992168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 372/754] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:13:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:06.994414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797186f8-2563-4bb5-85ff-3e2a00d5465e None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:07.095243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:07.117945 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797186f8-2563-4bb5-85ff-3e2a00d5465e None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:07.122151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075e9c6-a968-4f3b-9a2c-ed5fa270da3c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:07.161923 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe88d1-48e8-4393-9657-656dbde55c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:07.162565 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe88d1-48e8-4393-9657-656dbde55c1c None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:07.163166 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe88d1-48e8-4393-9657-656dbde55c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:07.163448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe88d1-48e8-4393-9657-656dbde55c1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:07.164412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe88d1-48e8-4393-9657-656dbde55c1c None admin] RBAC: Adding query filter params (interface=admin, service_id=4023d4c1025e4a47bd72e186f27b543c, region_id=RegionOne) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:07.164731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe88d1-48e8-4393-9657-656dbde55c1c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:07.166109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe88d1-48e8-4393-9657-656dbde55c1c None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:07.176990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 383/755] 10.222.0.20 () {56 vars in 1249 bytes} [Wed Oct 19 02:13:06 2022] GET /identity/v3/endpoints?service_id=4023d4c1025e4a47bd72e186f27b543c&interface=admin®ion_id=RegionOne => generated 193 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:07.179019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe88d1-48e8-4393-9657-656dbde55c1c None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:07.241533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075e9c6-a968-4f3b-9a2c-ed5fa270da3c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:07.275845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241092e2-d7bf-45b5-9db3-3c44786c1d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:07.276427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241092e2-d7bf-45b5-9db3-3c44786c1d27 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:07.276699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241092e2-d7bf-45b5-9db3-3c44786c1d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:07.276964 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241092e2-d7bf-45b5-9db3-3c44786c1d27 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:07.277651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241092e2-d7bf-45b5-9db3-3c44786c1d27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:07.279110 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241092e2-d7bf-45b5-9db3-3c44786c1d27 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:07.283598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe88d1-48e8-4393-9657-656dbde55c1c None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:07.319412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46155ff6-3449-43ce-9e36-ad8d73ef6ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:07.320024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46155ff6-3449-43ce-9e36-ad8d73ef6ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:07.320296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46155ff6-3449-43ce-9e36-ad8d73ef6ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:07.320562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46155ff6-3449-43ce-9e36-ad8d73ef6ce5 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:07.321179 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46155ff6-3449-43ce-9e36-ad8d73ef6ce5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:07.322561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.sql.core [None req-241092e2-d7bf-45b5-9db3-3c44786c1d27 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 19 02:13:07.322561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 19 02:13:07.322561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [parameters: {'user_id': 'b075629526cb48de9002416797770baf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 19 02:13:07.322561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81616) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 19 02:13:07.322861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46155ff6-3449-43ce-9e36-ad8d73ef6ce5 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:07.335273 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 384/756] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 19 02:13:07 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-241092e2-d7bf-45b5-9db3-3c44786c1d27 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application conn.query(q) Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result.read() Oct 19 02:13:07.335478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 02:13:07.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application next(self.gen) Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application yield resource Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application next(self.gen) Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.session.rollback() Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.force_reraise() Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise self.value Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application session.commit() Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 19 02:13:07.339215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._prepare_impl() Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.session.flush() Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._flush(objects) Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application compat.raise_( Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application flush_context.execute() Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rec.execute(self) Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 19 02:13:07.340386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 02:13:07.341268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application conn.query(q) Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result.read() Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 02:13:07.351111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b075629526cb48de9002416797770baf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:07.352725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 19 02:13:07.354309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 19 02:13:07.355810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 19 02:13:07.355810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 19 02:13:07.355810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 19 02:13:07.355810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 19 02:13:07.355810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 19 02:13:07.355810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:07.356376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2030465-c0c9-4c1a-963c-217fa9370c58 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:07.356376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2030465-c0c9-4c1a-963c-217fa9370c58 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:07.356376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2030465-c0c9-4c1a-963c-217fa9370c58 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:07.356376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 385/757] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:07.356694 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 373/758] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Oct 19 02:13:07 2022] POST /identity/v3/users => generated 160 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:07.356778 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2030465-c0c9-4c1a-963c-217fa9370c58 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:07.356860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74add1a2-1a7c-4a41-b601-e9b7429bf725 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:07.356860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74add1a2-1a7c-4a41-b601-e9b7429bf725 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:07.356860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74add1a2-1a7c-4a41-b601-e9b7429bf725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:07.371508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:07.376960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-74add1a2-1a7c-4a41-b601-e9b7429bf725 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:07.378704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-74add1a2-1a7c-4a41-b601-e9b7429bf725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:07.427845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:07.470225 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 374/759] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:07 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:07.471705 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598800f5-cdcc-4a15-8b27-b73a22525c79 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:07.471993 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598800f5-cdcc-4a15-8b27-b73a22525c79 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:07.472097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2030465-c0c9-4c1a-963c-217fa9370c58 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:07.472263 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598800f5-cdcc-4a15-8b27-b73a22525c79 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:07.476578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 375/760] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:07.482592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aacb4-ed71-4939-b59b-029e8a28e5ed None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:07.482951 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aacb4-ed71-4939-b59b-029e8a28e5ed None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:07.484261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aacb4-ed71-4939-b59b-029e8a28e5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:07.503737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-d51aacb4-ed71-4939-b59b-029e8a28e5ed None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:07.505051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d51aacb4-ed71-4939-b59b-029e8a28e5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:07.522282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2d792-5bbc-4845-a5fc-9428b3ea897f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:07.524317 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2d792-5bbc-4845-a5fc-9428b3ea897f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:07.524317 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2d792-5bbc-4845-a5fc-9428b3ea897f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:07.524317 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2d792-5bbc-4845-a5fc-9428b3ea897f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:07.533798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2d792-5bbc-4845-a5fc-9428b3ea897f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:07.535267 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e2d792-5bbc-4845-a5fc-9428b3ea897f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-b1e2d792-5bbc-4845-a5fc-9428b3ea897f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:13:07.548899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:07.550836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:07.552045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:07.552045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:07.552045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:13:07.552045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:13:07.552045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:13:07.552045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:13:07.552045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:13:07.552045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:13:07.552045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 19 02:13:07.552045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:07.552045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 386/761] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:13:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:07.558335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2d792-5bbc-4845-a5fc-9428b3ea897f None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:07.590394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 376/762] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:07 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:13:07.654407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:07.671450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e2d792-5bbc-4845-a5fc-9428b3ea897f None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:07.707807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9de24-e957-4db0-b019-1b9d8b8cc9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:07.708725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9de24-e957-4db0-b019-1b9d8b8cc9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:07.709227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9de24-e957-4db0-b019-1b9d8b8cc9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:07.709683 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9de24-e957-4db0-b019-1b9d8b8cc9f9 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:07.710594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9de24-e957-4db0-b019-1b9d8b8cc9f9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:07.711114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9de24-e957-4db0-b019-1b9d8b8cc9f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:07.712769 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9de24-e957-4db0-b019-1b9d8b8cc9f9 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:07.736200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aacb4-ed71-4939-b59b-029e8a28e5ed None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:07.751488 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 387/763] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Oct 19 02:13:07 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:07.855003 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aacb4-ed71-4939-b59b-029e8a28e5ed None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:07.856971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:07.863817 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9de24-e957-4db0-b019-1b9d8b8cc9f9 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:07.902262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7ff5-15a0-476d-be1b-6aeb7a33bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:07.903062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7ff5-15a0-476d-be1b-6aeb7a33bc58 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:07.903343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7ff5-15a0-476d-be1b-6aeb7a33bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:07.903712 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da7ff5-15a0-476d-be1b-6aeb7a33bc58 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:07.904322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da7ff5-15a0-476d-be1b-6aeb7a33bc58 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:07.906003 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da7ff5-15a0-476d-be1b-6aeb7a33bc58 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-d3da7ff5-15a0-476d-be1b-6aeb7a33bc58 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:07.917728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:07.918810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:07.919641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 377/764] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:13:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:13:07.925910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7ff5-15a0-476d-be1b-6aeb7a33bc58 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:07.985444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9de24-e957-4db0-b019-1b9d8b8cc9f9 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:08.026191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccab9bc-3aba-4ef9-b641-44e228a27b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:08.027264 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccab9bc-3aba-4ef9-b641-44e228a27b9d None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:08.027264 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccab9bc-3aba-4ef9-b641-44e228a27b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:08.027767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccab9bc-3aba-4ef9-b641-44e228a27b9d None admin] PATH_INFO: `/v3/services/image` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:08.028353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccab9bc-3aba-4ef9-b641-44e228a27b9d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:08.030221 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccab9bc-3aba-4ef9-b641-44e228a27b9d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:08.033605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-7ccab9bc-3aba-4ef9-b641-44e228a27b9d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:08.040218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:08.041210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:08.042920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 388/765] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Oct 19 02:13:07 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:13:08.043966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da7ff5-15a0-476d-be1b-6aeb7a33bc58 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:08.047617 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccab9bc-3aba-4ef9-b641-44e228a27b9d None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:08.094400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d98a8-0d87-449e-add2-fddac429b6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:08.095018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d98a8-0d87-449e-add2-fddac429b6fd None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:08.095294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d98a8-0d87-449e-add2-fddac429b6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:08.095561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d98a8-0d87-449e-add2-fddac429b6fd None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:08.096195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d98a8-0d87-449e-add2-fddac429b6fd None admin] RBAC: Adding query filter params (name=compute) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:08.096494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d98a8-0d87-449e-add2-fddac429b6fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:08.097853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d98a8-0d87-449e-add2-fddac429b6fd None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:08.111984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 378/766] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:13:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:08.113186 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d98a8-0d87-449e-add2-fddac429b6fd None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:08.149623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:08.167714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccab9bc-3aba-4ef9-b641-44e228a27b9d None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:08.204058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3761e00-270c-439c-8bf9-e222cfc3c1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:08.204658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3761e00-270c-439c-8bf9-e222cfc3c1fa None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:08.204943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3761e00-270c-439c-8bf9-e222cfc3c1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:08.205215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3761e00-270c-439c-8bf9-e222cfc3c1fa None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:08.205833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3761e00-270c-439c-8bf9-e222cfc3c1fa None admin] RBAC: Adding query filter params (name=image) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:08.206208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3761e00-270c-439c-8bf9-e222cfc3c1fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:08.207661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3761e00-270c-439c-8bf9-e222cfc3c1fa None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:08.212538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 389/767] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:08.218834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3761e00-270c-439c-8bf9-e222cfc3c1fa None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:08.228934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d98a8-0d87-449e-add2-fddac429b6fd None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:08.266086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf812b4f-ec6e-4dad-aca3-b9ccd4afd54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:08.266708 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf812b4f-ec6e-4dad-aca3-b9ccd4afd54d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:08.266984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf812b4f-ec6e-4dad-aca3-b9ccd4afd54d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:08.267253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf812b4f-ec6e-4dad-aca3-b9ccd4afd54d None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:08.267897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf812b4f-ec6e-4dad-aca3-b9ccd4afd54d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:08.268196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf812b4f-ec6e-4dad-aca3-b9ccd4afd54d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:08.269596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf812b4f-ec6e-4dad-aca3-b9ccd4afd54d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:08.275489 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 379/768] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Oct 19 02:13:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:08.281268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf812b4f-ec6e-4dad-aca3-b9ccd4afd54d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:08.388620 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:08.395682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf812b4f-ec6e-4dad-aca3-b9ccd4afd54d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:08.398816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:08.431112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5cd86b-fc87-4c52-854c-c5a7ce25cc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:08.431794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cd86b-fc87-4c52-854c-c5a7ce25cc9f None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:08.432125 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cd86b-fc87-4c52-854c-c5a7ce25cc9f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:08.432397 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cd86b-fc87-4c52-854c-c5a7ce25cc9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:08.433142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5cd86b-fc87-4c52-854c-c5a7ce25cc9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:08.434512 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5cd86b-fc87-4c52-854c-c5a7ce25cc9f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:08.437137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3761e00-270c-439c-8bf9-e222cfc3c1fa None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:08.471167 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d6af5-a56e-403e-8268-7608cf26090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:08.471767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d6af5-a56e-403e-8268-7608cf26090a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:08.472043 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d6af5-a56e-403e-8268-7608cf26090a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:08.472407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d6af5-a56e-403e-8268-7608cf26090a None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:08.472955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d6af5-a56e-403e-8268-7608cf26090a None admin] RBAC: Adding query filter params (type=image) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:08.473251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d6af5-a56e-403e-8268-7608cf26090a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:08.474624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d6af5-a56e-403e-8268-7608cf26090a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:08.479382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 390/769] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:08.485180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d6af5-a56e-403e-8268-7608cf26090a None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:08.508913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 380/770] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:13:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:13:08.598707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d6af5-a56e-403e-8268-7608cf26090a None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:08.628525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:08.644869 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8823dbe-1132-4eda-8b5c-2ad69e58ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:08.645505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8823dbe-1132-4eda-8b5c-2ad69e58ed7f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:08.645843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8823dbe-1132-4eda-8b5c-2ad69e58ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:08.646211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8823dbe-1132-4eda-8b5c-2ad69e58ed7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:08.646989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8823dbe-1132-4eda-8b5c-2ad69e58ed7f None admin] RBAC: Adding query filter params (interface=public, service_id=a7d345004b9047f6ac319380532a5e98, region_id=RegionOne) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:08.647362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8823dbe-1132-4eda-8b5c-2ad69e58ed7f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:08.648879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8823dbe-1132-4eda-8b5c-2ad69e58ed7f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:08.659461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 391/771] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 19 02:13:08 2022] GET /identity/v3/endpoints?service_id=a7d345004b9047f6ac319380532a5e98&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:08.661201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5cd86b-fc87-4c52-854c-c5a7ce25cc9f None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:08.771261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5cd86b-fc87-4c52-854c-c5a7ce25cc9f None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:08.806837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeee153-bf28-453c-b989-ea5ede9aceec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:08.807446 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeee153-bf28-453c-b989-ea5ede9aceec None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:08.807718 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeee153-bf28-453c-b989-ea5ede9aceec None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:08.807986 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeee153-bf28-453c-b989-ea5ede9aceec None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:08.808584 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeee153-bf28-453c-b989-ea5ede9aceec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:08.809939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeee153-bf28-453c-b989-ea5ede9aceec None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:08.816976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 381/772] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 19 02:13:08 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:13:09.588638 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3e0dd-7eee-472d-88a1-6d2404e7ce3d None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:09.589154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3e0dd-7eee-472d-88a1-6d2404e7ce3d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:09.589426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3e0dd-7eee-472d-88a1-6d2404e7ce3d None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:09.590908 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 392/773] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:09.596694 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77a8ba-da41-4637-9cca-ddf46c0c4378 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:09.596982 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77a8ba-da41-4637-9cca-ddf46c0c4378 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:09.597251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77a8ba-da41-4637-9cca-ddf46c0c4378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:09.615071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2e77a8ba-da41-4637-9cca-ddf46c0c4378 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:09.616237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2e77a8ba-da41-4637-9cca-ddf46c0c4378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:09.626331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:09.685538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 382/774] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:09 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:09.947527 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3e0dd-7eee-472d-88a1-6d2404e7ce3d None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:10.053206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3e0dd-7eee-472d-88a1-6d2404e7ce3d None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:10.086958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5385387-7eae-4817-8156-1123a6aea7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:10.087573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5385387-7eae-4817-8156-1123a6aea7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:10.087845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5385387-7eae-4817-8156-1123a6aea7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:10.088108 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5385387-7eae-4817-8156-1123a6aea7c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:10.088677 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5385387-7eae-4817-8156-1123a6aea7c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:10.090398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5385387-7eae-4817-8156-1123a6aea7c7 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-d5385387-7eae-4817-8156-1123a6aea7c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:10.106128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:10.107232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:10.108292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 393/775] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Oct 19 02:13:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:10.110705 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77a8ba-da41-4637-9cca-ddf46c0c4378 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:10.209497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77a8ba-da41-4637-9cca-ddf46c0c4378 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:10.242264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf4c78-ccd6-4e77-b915-b60be59d690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:10.242870 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf4c78-ccd6-4e77-b915-b60be59d690d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:10.243148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf4c78-ccd6-4e77-b915-b60be59d690d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:10.243419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf4c78-ccd6-4e77-b915-b60be59d690d None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:10.244042 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf4c78-ccd6-4e77-b915-b60be59d690d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:10.244340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf4c78-ccd6-4e77-b915-b60be59d690d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:10.245685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf4c78-ccd6-4e77-b915-b60be59d690d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:10.251308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 383/776] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:13:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:10.256839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5385387-7eae-4817-8156-1123a6aea7c7 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:10.311300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:10.361250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5385387-7eae-4817-8156-1123a6aea7c7 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:10.393733 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd057e-8f81-4650-9932-152b532c9c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:10.394365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd057e-8f81-4650-9932-152b532c9c1e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:10.394640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd057e-8f81-4650-9932-152b532c9c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:10.394904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd057e-8f81-4650-9932-152b532c9c1e None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:10.395521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd057e-8f81-4650-9932-152b532c9c1e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:10.395814 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd057e-8f81-4650-9932-152b532c9c1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:10.397155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd057e-8f81-4650-9932-152b532c9c1e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:10.402154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 394/777] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Oct 19 02:13:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:10.408558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf4c78-ccd6-4e77-b915-b60be59d690d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:10.419520 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:10.510735 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf4c78-ccd6-4e77-b915-b60be59d690d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:10.544390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad90bc-002c-4e72-82fd-b09c481679c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:10.544999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad90bc-002c-4e72-82fd-b09c481679c8 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:10.545275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad90bc-002c-4e72-82fd-b09c481679c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:10.545546 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad90bc-002c-4e72-82fd-b09c481679c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:10.546828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad90bc-002c-4e72-82fd-b09c481679c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:10.548179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad90bc-002c-4e72-82fd-b09c481679c8 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:10.654084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 384/778] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:13:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:13:10.771633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684772-d28f-461c-aa03-488574687358 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:10.772131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684772-d28f-461c-aa03-488574687358 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:10.772131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27684772-d28f-461c-aa03-488574687358 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:10.773558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 395/779] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:10.779459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78356148-1d74-4977-bded-c452450c3ae3 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:10.779785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78356148-1d74-4977-bded-c452450c3ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:10.780016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78356148-1d74-4977-bded-c452450c3ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:10.799533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-78356148-1d74-4977-bded-c452450c3ae3 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:10.800753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-78356148-1d74-4977-bded-c452450c3ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:10.871594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:10.884112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 385/780] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:10.965574 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c188a-bbb1-4cc9-95e6-1e273081719c None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:10.965882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c188a-bbb1-4cc9-95e6-1e273081719c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:10.966138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c188a-bbb1-4cc9-95e6-1e273081719c None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:10.967596 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 396/781] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:10.972607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd5306-e703-48a5-ba78-ae255bc40b99 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:10.972891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd5306-e703-48a5-ba78-ae255bc40b99 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:10.973155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd5306-e703-48a5-ba78-ae255bc40b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:10.991040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-44dd5306-e703-48a5-ba78-ae255bc40b99 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:10.992200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-44dd5306-e703-48a5-ba78-ae255bc40b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:11.057712 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 386/782] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:11.152534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c188a-bbb1-4cc9-95e6-1e273081719c None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:11.254639 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c188a-bbb1-4cc9-95e6-1e273081719c None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:11.270682 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:11.288810 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce432e-3331-40b2-ad7f-8d68b163f1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:11.289432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce432e-3331-40b2-ad7f-8d68b163f1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:11.289711 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce432e-3331-40b2-ad7f-8d68b163f1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:11.289974 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce432e-3331-40b2-ad7f-8d68b163f1e0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:11.300552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce432e-3331-40b2-ad7f-8d68b163f1e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-ebce432e-3331-40b2-ad7f-8d68b163f1e0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:13:11.305295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:11.306440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 02:13:11.307288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:11.310408 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 397/783] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 19 02:13:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:11.312475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd5306-e703-48a5-ba78-ae255bc40b99 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:11.321993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce432e-3331-40b2-ad7f-8d68b163f1e0 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:11.415052 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd5306-e703-48a5-ba78-ae255bc40b99 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:11.422476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce432e-3331-40b2-ad7f-8d68b163f1e0 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:11.439548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:11.450501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212ac7b-d4be-4ccf-a90f-140aacc0039a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:11.451112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212ac7b-d4be-4ccf-a90f-140aacc0039a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:11.451398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212ac7b-d4be-4ccf-a90f-140aacc0039a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:11.451671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212ac7b-d4be-4ccf-a90f-140aacc0039a None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:11.452876 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212ac7b-d4be-4ccf-a90f-140aacc0039a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:11.453172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212ac7b-d4be-4ccf-a90f-140aacc0039a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:11.454562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212ac7b-d4be-4ccf-a90f-140aacc0039a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:11.457206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c238cd-f6dd-47fa-abec-bb45b0fa6fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:11.459992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c238cd-f6dd-47fa-abec-bb45b0fa6fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:11.459992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c238cd-f6dd-47fa-abec-bb45b0fa6fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:11.459992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c238cd-f6dd-47fa-abec-bb45b0fa6fd7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:11.459992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c238cd-f6dd-47fa-abec-bb45b0fa6fd7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:11.460309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c238cd-f6dd-47fa-abec-bb45b0fa6fd7 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:11.463463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 387/784] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Oct 19 02:13:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:11.469114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212ac7b-d4be-4ccf-a90f-140aacc0039a None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-28c238cd-f6dd-47fa-abec-bb45b0fa6fd7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:11.469348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:11.470342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:11.471170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 398/785] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Oct 19 02:13:11 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:13:11.476133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c238cd-f6dd-47fa-abec-bb45b0fa6fd7 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:11.517256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:11.576578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c238cd-f6dd-47fa-abec-bb45b0fa6fd7 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:11.579727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212ac7b-d4be-4ccf-a90f-140aacc0039a None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:11.610634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b5eb0-1b1a-459b-b9d4-1ded78945c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:11.611226 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b5eb0-1b1a-459b-b9d4-1ded78945c74 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:11.611500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b5eb0-1b1a-459b-b9d4-1ded78945c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:11.611767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b5eb0-1b1a-459b-b9d4-1ded78945c74 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:11.612378 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b5eb0-1b1a-459b-b9d4-1ded78945c74 None admin] RBAC: Adding query filter params (name=image) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:11.612676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b5eb0-1b1a-459b-b9d4-1ded78945c74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:11.614014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b5eb0-1b1a-459b-b9d4-1ded78945c74 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:11.614454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3085d-8020-4bdf-92a9-5b507adc2021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:11.615077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3085d-8020-4bdf-92a9-5b507adc2021 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:11.615356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3085d-8020-4bdf-92a9-5b507adc2021 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:11.615635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3085d-8020-4bdf-92a9-5b507adc2021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:11.618750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 399/786] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:11.624294 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b5eb0-1b1a-459b-b9d4-1ded78945c74 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:11.626439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3085d-8020-4bdf-92a9-5b507adc2021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:11.627875 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3085d-8020-4bdf-92a9-5b507adc2021 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-edf3085d-8020-4bdf-92a9-5b507adc2021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:13:11.645657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:11.646625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:13:11.647585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:13:11.647585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:13:11.647585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:11.647585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 388/787] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:13:11.652350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3085d-8020-4bdf-92a9-5b507adc2021 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:11.692718 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:11.732755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b5eb0-1b1a-459b-b9d4-1ded78945c74 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:11.735836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:11.756152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3085d-8020-4bdf-92a9-5b507adc2021 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:11.769729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bb88d-d81e-42a9-821a-366df0bb9858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:11.770765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bb88d-d81e-42a9-821a-366df0bb9858 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:11.771293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bb88d-d81e-42a9-821a-366df0bb9858 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:11.772044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125bb88d-d81e-42a9-821a-366df0bb9858 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:11.773326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125bb88d-d81e-42a9-821a-366df0bb9858 None admin] RBAC: Adding query filter params (type=image) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:11.774453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125bb88d-d81e-42a9-821a-366df0bb9858 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:11.775811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125bb88d-d81e-42a9-821a-366df0bb9858 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:11.781740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 400/788] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:11.787272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bb88d-d81e-42a9-821a-366df0bb9858 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:11.795302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9c3f2-e049-46c0-9718-4ab1626bdf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:11.795937 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9c3f2-e049-46c0-9718-4ab1626bdf15 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:11.796230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9c3f2-e049-46c0-9718-4ab1626bdf15 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:11.796499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9c3f2-e049-46c0-9718-4ab1626bdf15 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:11.797144 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9c3f2-e049-46c0-9718-4ab1626bdf15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:11.797441 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9c3f2-e049-46c0-9718-4ab1626bdf15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:11.798821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9c3f2-e049-46c0-9718-4ab1626bdf15 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:11.812163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 389/789] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:11.817411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9c3f2-e049-46c0-9718-4ab1626bdf15 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:11.890249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125bb88d-d81e-42a9-821a-366df0bb9858 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:11.916112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:11.920508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9c3f2-e049-46c0-9718-4ab1626bdf15 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:11.924295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804607d-7bb2-4ddc-845f-fea9654c9c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:11.924899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804607d-7bb2-4ddc-845f-fea9654c9c5c None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:11.945851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804607d-7bb2-4ddc-845f-fea9654c9c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:11.945851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804607d-7bb2-4ddc-845f-fea9654c9c5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:11.945851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804607d-7bb2-4ddc-845f-fea9654c9c5c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:11.945851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804607d-7bb2-4ddc-845f-fea9654c9c5c None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:11.955428 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af2b8d-8422-4f2e-b76d-ed838023773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:11.956056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af2b8d-8422-4f2e-b76d-ed838023773c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:11.956336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af2b8d-8422-4f2e-b76d-ed838023773c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:11.956603 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af2b8d-8422-4f2e-b76d-ed838023773c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:11.963490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af2b8d-8422-4f2e-b76d-ed838023773c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:11.965296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af2b8d-8422-4f2e-b76d-ed838023773c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-c1af2b8d-8422-4f2e-b76d-ed838023773c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:13:11.977583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:11.978565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:11.979491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:11.979491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:11.979491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:13:11.979491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:13:11.979491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:13:11.979491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:13:11.979491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:13:11.979491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:13:11.979491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:13:11.979491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:11.980470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 390/790] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 19 02:13:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:13:11.984796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af2b8d-8422-4f2e-b76d-ed838023773c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:12.038072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 401/791] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:13:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:13:12.088780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af2b8d-8422-4f2e-b76d-ed838023773c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:12.138103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645332b-c6cb-4c51-a680-b90be7ee4198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:12.138723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645332b-c6cb-4c51-a680-b90be7ee4198 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:12.139002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645332b-c6cb-4c51-a680-b90be7ee4198 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:12.139277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645332b-c6cb-4c51-a680-b90be7ee4198 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:12.140604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645332b-c6cb-4c51-a680-b90be7ee4198 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:12.140852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645332b-c6cb-4c51-a680-b90be7ee4198 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:12.142314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645332b-c6cb-4c51-a680-b90be7ee4198 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:12.166119 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 391/792] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 19 02:13:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:12.167998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804607d-7bb2-4ddc-845f-fea9654c9c5c None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:12.197037 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:12.273253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804607d-7bb2-4ddc-845f-fea9654c9c5c None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:12.308314 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb390a9-9fde-4650-96b8-d635d2502a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:12.308942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb390a9-9fde-4650-96b8-d635d2502a1a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:12.309224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb390a9-9fde-4650-96b8-d635d2502a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:12.309490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb390a9-9fde-4650-96b8-d635d2502a1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:12.320287 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb390a9-9fde-4650-96b8-d635d2502a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:12.321724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb390a9-9fde-4650-96b8-d635d2502a1a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-4bb390a9-9fde-4650-96b8-d635d2502a1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:13:12.336160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:12.337147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:13:12.338119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:13:12.338119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:13:12.338119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:12.338119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 402/793] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:12.344629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645332b-c6cb-4c51-a680-b90be7ee4198 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:12.367258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:12.456183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645332b-c6cb-4c51-a680-b90be7ee4198 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:12.490072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3b44a-615a-407a-a6c4-5408c8857139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:12.490683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3b44a-615a-407a-a6c4-5408c8857139 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:12.490959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3b44a-615a-407a-a6c4-5408c8857139 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:12.491228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3b44a-615a-407a-a6c4-5408c8857139 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:12.491869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3b44a-615a-407a-a6c4-5408c8857139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:12.492167 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3b44a-615a-407a-a6c4-5408c8857139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:12.493528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3b44a-615a-407a-a6c4-5408c8857139 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:12.509775 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 392/794] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:12.511016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb390a9-9fde-4650-96b8-d635d2502a1a None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:12.589504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:12.611901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb390a9-9fde-4650-96b8-d635d2502a1a None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:12.645766 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59891c79-aad3-4d82-b9e8-0b829f8d8d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:12.646431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891c79-aad3-4d82-b9e8-0b829f8d8d5e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:12.646708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891c79-aad3-4d82-b9e8-0b829f8d8d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:12.651262 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59891c79-aad3-4d82-b9e8-0b829f8d8d5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:12.653632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59891c79-aad3-4d82-b9e8-0b829f8d8d5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:12.655108 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59891c79-aad3-4d82-b9e8-0b829f8d8d5e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-59891c79-aad3-4d82-b9e8-0b829f8d8d5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:13:12.666410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:12.667312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:12.668209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:13:12.668209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:13:12.668209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:13:12.668209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:13:12.668209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:13:12.668209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:13:12.668209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:13:12.668209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:12.668209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 403/795] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:13:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:13:12.673359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3b44a-615a-407a-a6c4-5408c8857139 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:12.750422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:12.782254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3b44a-615a-407a-a6c4-5408c8857139 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:12.817069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ef50e-582d-4300-a06e-0f0afea1dbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:12.817680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ef50e-582d-4300-a06e-0f0afea1dbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:12.817959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ef50e-582d-4300-a06e-0f0afea1dbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:12.818255 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ef50e-582d-4300-a06e-0f0afea1dbb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:12.818941 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ef50e-582d-4300-a06e-0f0afea1dbb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:12.819242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ef50e-582d-4300-a06e-0f0afea1dbb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:12.820661 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ef50e-582d-4300-a06e-0f0afea1dbb7 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:12.832792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 393/796] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:13:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:12.838584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59891c79-aad3-4d82-b9e8-0b829f8d8d5e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:12.845799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce36bd7-ab03-40a7-99a6-2bc23f09e247 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:12.846115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce36bd7-ab03-40a7-99a6-2bc23f09e247 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:12.846385 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce36bd7-ab03-40a7-99a6-2bc23f09e247 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:12.847838 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 394/797] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:13:12.852760 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fd469-567f-48f7-ab97-3aba6c8887e8 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:12.853036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fd469-567f-48f7-ab97-3aba6c8887e8 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:12.853306 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fd469-567f-48f7-ab97-3aba6c8887e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:12.871762 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-df3fd469-567f-48f7-ab97-3aba6c8887e8 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:12.872927 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-df3fd469-567f-48f7-ab97-3aba6c8887e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:12.938735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59891c79-aad3-4d82-b9e8-0b829f8d8d5e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:12.951019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 395/798] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:13:12.974619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2093026-f0be-469b-86c9-ba0d32635634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:12.975266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2093026-f0be-469b-86c9-ba0d32635634 None admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:12.975550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2093026-f0be-469b-86c9-ba0d32635634 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:12.975821 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2093026-f0be-469b-86c9-ba0d32635634 None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/0abf52399a344157ac4e0f9a8212164e/roles/e95a043c68e440b2a961391d6e6e625b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:12.977920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2093026-f0be-469b-86c9-ba0d32635634 None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=0abf52399a344157ac4e0f9a8212164e, role_id=e95a043c68e440b2a961391d6e6e625b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:12.979376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2093026-f0be-469b-86c9-ba0d32635634 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:12.982403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:13.087969 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 404/799] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:13:12 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/0abf52399a344157ac4e0f9a8212164e/roles/e95a043c68e440b2a961391d6e6e625b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:13:13.213952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fd469-567f-48f7-ab97-3aba6c8887e8 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:13.278721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:13.317252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fd469-567f-48f7-ab97-3aba6c8887e8 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:13.351051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf3acb-b42c-44d8-ae92-109e5b3d3645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:13.351651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf3acb-b42c-44d8-ae92-109e5b3d3645 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:13.351855 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf3acb-b42c-44d8-ae92-109e5b3d3645 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:13.352126 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf3acb-b42c-44d8-ae92-109e5b3d3645 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:13.353267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf3acb-b42c-44d8-ae92-109e5b3d3645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:13.354710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf3acb-b42c-44d8-ae92-109e5b3d3645 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:13.356431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 396/800] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:13.362745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2093026-f0be-469b-86c9-ba0d32635634 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:13.460649 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2093026-f0be-469b-86c9-ba0d32635634 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:13.494198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f885ab-f0f8-42af-9c82-a001a586c5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:13.494789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f885ab-f0f8-42af-9c82-a001a586c5b1 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:13.495061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f885ab-f0f8-42af-9c82-a001a586c5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:13.495326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f885ab-f0f8-42af-9c82-a001a586c5b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:13.495975 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f885ab-f0f8-42af-9c82-a001a586c5b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:13.497378 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f885ab-f0f8-42af-9c82-a001a586c5b1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:13.606144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 405/801] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:13:13 2022] POST /identity/v3/projects => generated 311 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:13:14.235934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8d91e-206d-49f1-b49e-cb87c154854c None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:14.236288 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8d91e-206d-49f1-b49e-cb87c154854c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:14.236488 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8d91e-206d-49f1-b49e-cb87c154854c None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:14.237967 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 397/802] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:14.242942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cc2b-a1ad-4c6e-80ac-3ee61d578ef5 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:14.243381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cc2b-a1ad-4c6e-80ac-3ee61d578ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:14.243560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0cc2b-a1ad-4c6e-80ac-3ee61d578ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:14.261563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-21c0cc2b-a1ad-4c6e-80ac-3ee61d578ef5 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:14.262764 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-21c0cc2b-a1ad-4c6e-80ac-3ee61d578ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:14.307958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:14.331230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 406/803] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:14.609862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8d91e-206d-49f1-b49e-cb87c154854c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:14.714217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8d91e-206d-49f1-b49e-cb87c154854c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:14.742786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:14.749090 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27ba48-bfbe-45af-880f-4f9bf5750866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:14.750247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27ba48-bfbe-45af-880f-4f9bf5750866 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:14.750525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27ba48-bfbe-45af-880f-4f9bf5750866 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:14.750795 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27ba48-bfbe-45af-880f-4f9bf5750866 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:14.751484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27ba48-bfbe-45af-880f-4f9bf5750866 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:14.752954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27ba48-bfbe-45af-880f-4f9bf5750866 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:14.796369 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.sql.core [None req-dd27ba48-bfbe-45af-880f-4f9bf5750866 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 02:13:14.796369 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 02:13:14.796369 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [parameters: {'id': '1b06c8118fbd4ebbbbe2f4691dbd2b92', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 19 02:13:14.796369 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81616) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-dd27ba48-bfbe-45af-880f-4f9bf5750866 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application conn.query(q) Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 02:13:14.803615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result.read() Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 19 02:13:14.807060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application next(self.gen) Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application yield resource Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application next(self.gen) Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.session.rollback() Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.force_reraise() Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise self.value Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application session.commit() Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 19 02:13:14.808060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._prepare_impl() Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.session.flush() Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._flush(objects) Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application compat.raise_( Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application flush_context.execute() Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rec.execute(self) Oct 19 02:13:14.808939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 19 02:13:14.810882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result = self._query(query) Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application conn.query(q) Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application result.read() Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 19 02:13:14.812683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [parameters: {'id': '1b06c8118fbd4ebbbbe2f4691dbd2b92', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 19 02:13:14.813610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:14.814435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:14.815325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 398/804] 10.222.0.20 () {60 vars in 1137 bytes} [Wed Oct 19 02:13:14 2022] POST /identity/v3/domains => generated 190 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:14.816022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0cc2b-a1ad-4c6e-80ac-3ee61d578ef5 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:14.915764 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0cc2b-a1ad-4c6e-80ac-3ee61d578ef5 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:14.948819 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10499a39-58cb-4128-8fa3-e54de3a53393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:14.949501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10499a39-58cb-4128-8fa3-e54de3a53393 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:14.949781 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10499a39-58cb-4128-8fa3-e54de3a53393 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:14.950064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10499a39-58cb-4128-8fa3-e54de3a53393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:14.960419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10499a39-58cb-4128-8fa3-e54de3a53393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:14.961853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10499a39-58cb-4128-8fa3-e54de3a53393 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-10499a39-58cb-4128-8fa3-e54de3a53393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:13:14.976300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:14.977222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:13:14.978708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:13:14.978708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:13:14.978708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:14.978708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 407/805] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:14.983309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27ba48-bfbe-45af-880f-4f9bf5750866 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:15.079950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27ba48-bfbe-45af-880f-4f9bf5750866 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:15.120285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9480a6d4-4c39-4da4-8995-5bfa10e77814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:15.120922 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480a6d4-4c39-4da4-8995-5bfa10e77814 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:15.121199 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480a6d4-4c39-4da4-8995-5bfa10e77814 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:15.121467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480a6d4-4c39-4da4-8995-5bfa10e77814 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:15.122168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480a6d4-4c39-4da4-8995-5bfa10e77814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:15.122478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480a6d4-4c39-4da4-8995-5bfa10e77814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:15.123898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9480a6d4-4c39-4da4-8995-5bfa10e77814 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:15.146095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 399/806] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:15.236868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c1bfe-4091-43a7-b896-2660235e688f None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:15.237064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c1bfe-4091-43a7-b896-2660235e688f None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:15.237330 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c1bfe-4091-43a7-b896-2660235e688f None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:15.238929 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 408/807] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:15.244416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214e64-e813-428f-a884-74f557c9ce36 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:15.244702 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214e64-e813-428f-a884-74f557c9ce36 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:15.244968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd214e64-e813-428f-a884-74f557c9ce36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:15.264286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-cd214e64-e813-428f-a884-74f557c9ce36 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:15.265983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cd214e64-e813-428f-a884-74f557c9ce36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:15.318278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:15.336029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 400/808] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:15.600049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c1bfe-4091-43a7-b896-2660235e688f None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:15.630395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:15.702245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c1bfe-4091-43a7-b896-2660235e688f None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:15.737326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d73f66-fe63-47b1-8049-1a33cc6c0e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:15.737947 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d73f66-fe63-47b1-8049-1a33cc6c0e11 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:15.738264 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d73f66-fe63-47b1-8049-1a33cc6c0e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:15.738539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d73f66-fe63-47b1-8049-1a33cc6c0e11 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:15.748268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d73f66-fe63-47b1-8049-1a33cc6c0e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-b2d73f66-fe63-47b1-8049-1a33cc6c0e11 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:13:15.753553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:15.754495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:15.755327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 409/809] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Oct 19 02:13:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:15.760669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd214e64-e813-428f-a884-74f557c9ce36 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:15.797997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242069c9-c008-4728-818f-6f7a8a5d3070 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:15.798301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242069c9-c008-4728-818f-6f7a8a5d3070 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:15.798571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242069c9-c008-4728-818f-6f7a8a5d3070 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:15.800448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 410/810] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:13:15.805254 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5498f-5462-4a49-ba98-6ebce63dd9eb None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:15.805529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5498f-5462-4a49-ba98-6ebce63dd9eb None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:15.805792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5498f-5462-4a49-ba98-6ebce63dd9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:15.823611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e2c5498f-5462-4a49-ba98-6ebce63dd9eb None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:15.824770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e2c5498f-5462-4a49-ba98-6ebce63dd9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:15.857112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd214e64-e813-428f-a884-74f557c9ce36 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:15.889366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 411/811] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:13:15.890381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179feb4-fd21-4ff2-b367-9c44a1603c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:15.891012 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179feb4-fd21-4ff2-b367-9c44a1603c5c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:15.891291 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179feb4-fd21-4ff2-b367-9c44a1603c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:15.891561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179feb4-fd21-4ff2-b367-9c44a1603c5c None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:15.892214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179feb4-fd21-4ff2-b367-9c44a1603c5c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:15.892512 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179feb4-fd21-4ff2-b367-9c44a1603c5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:15.893857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179feb4-fd21-4ff2-b367-9c44a1603c5c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:15.906234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 401/812] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Oct 19 02:13:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:15.907296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5498f-5462-4a49-ba98-6ebce63dd9eb None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:15.927820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:16.001685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5498f-5462-4a49-ba98-6ebce63dd9eb None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:16.033220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e907-8058-44d6-9920-dee68479f81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:16.033832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e907-8058-44d6-9920-dee68479f81a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:16.034137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e907-8058-44d6-9920-dee68479f81a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:16.034408 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e907-8058-44d6-9920-dee68479f81a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:16.043840 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337e907-8058-44d6-9920-dee68479f81a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:16.045266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337e907-8058-44d6-9920-dee68479f81a None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-d337e907-8058-44d6-9920-dee68479f81a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:13:16.059608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:16.060783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:13:16.062472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:13:16.062472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:13:16.062472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:16.062472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 412/813] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:16.066957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179feb4-fd21-4ff2-b367-9c44a1603c5c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:16.113087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:16.163191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e907-8058-44d6-9920-dee68479f81a None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:16.166241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179feb4-fd21-4ff2-b367-9c44a1603c5c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:16.198660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c562a9a-af18-495b-95d2-3a1c3606b325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:16.199258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c562a9a-af18-495b-95d2-3a1c3606b325 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:16.199536 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c562a9a-af18-495b-95d2-3a1c3606b325 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:16.199808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c562a9a-af18-495b-95d2-3a1c3606b325 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:16.200447 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c562a9a-af18-495b-95d2-3a1c3606b325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:16.200748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c562a9a-af18-495b-95d2-3a1c3606b325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:16.202158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c562a9a-af18-495b-95d2-3a1c3606b325 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:16.213427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 402/814] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:16.219738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c562a9a-af18-495b-95d2-3a1c3606b325 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:16.233549 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:16.263498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e907-8058-44d6-9920-dee68479f81a None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:16.298464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ecf12-6be1-4626-8c68-7173d4a80d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:16.299323 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ecf12-6be1-4626-8c68-7173d4a80d4c None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:16.299596 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ecf12-6be1-4626-8c68-7173d4a80d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:16.299900 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ecf12-6be1-4626-8c68-7173d4a80d4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:16.301062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ecf12-6be1-4626-8c68-7173d4a80d4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:16.302531 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ecf12-6be1-4626-8c68-7173d4a80d4c None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:16.304084 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 413/815] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:16.309824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ecf12-6be1-4626-8c68-7173d4a80d4c None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:16.316676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c562a9a-af18-495b-95d2-3a1c3606b325 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:16.323966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:16.350152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ee49-5964-43aa-85bd-1466aa580b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:16.350970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ee49-5964-43aa-85bd-1466aa580b19 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:16.351352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ee49-5964-43aa-85bd-1466aa580b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:16.351625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0ee49-5964-43aa-85bd-1466aa580b19 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:16.360201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0ee49-5964-43aa-85bd-1466aa580b19 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:16.360948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0ee49-5964-43aa-85bd-1466aa580b19 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-e4e0ee49-5964-43aa-85bd-1466aa580b19 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:13:16.386778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:16.388239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:16.389334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:16.389334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:16.389334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:13:16.389334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:13:16.389334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:13:16.389334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:13:16.389334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:13:16.389334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:13:16.389334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 19 02:13:16.389334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:16.391101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 403/816] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Oct 19 02:13:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:16.394883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ee49-5964-43aa-85bd-1466aa580b19 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:16.418867 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ecf12-6be1-4626-8c68-7173d4a80d4c None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:16.451535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a08e4b-333b-4db0-a9b6-c86f521fbc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:16.452123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08e4b-333b-4db0-a9b6-c86f521fbc88 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:16.452429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08e4b-333b-4db0-a9b6-c86f521fbc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:16.453222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08e4b-333b-4db0-a9b6-c86f521fbc88 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:16.453860 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a08e4b-333b-4db0-a9b6-c86f521fbc88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:16.455304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a08e4b-333b-4db0-a9b6-c86f521fbc88 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:16.466145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:16.490607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0ee49-5964-43aa-85bd-1466aa580b19 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:16.523766 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc09411-69d2-4882-9314-c7bf565401f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:16.525435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc09411-69d2-4882-9314-c7bf565401f6 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:16.525435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc09411-69d2-4882-9314-c7bf565401f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:16.525435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc09411-69d2-4882-9314-c7bf565401f6 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:16.525759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc09411-69d2-4882-9314-c7bf565401f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:16.525887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc09411-69d2-4882-9314-c7bf565401f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:16.527372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc09411-69d2-4882-9314-c7bf565401f6 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:16.544006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 404/817] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Oct 19 02:13:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:13:16.550201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc09411-69d2-4882-9314-c7bf565401f6 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:16.582073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 414/818] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:13:16 2022] POST /identity/v3/users => generated 285 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:13:16.583330 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:16.652692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc09411-69d2-4882-9314-c7bf565401f6 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:16.689040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43edcd-1698-4668-9476-bca47bdebfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:16.689666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43edcd-1698-4668-9476-bca47bdebfbd None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:16.689948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43edcd-1698-4668-9476-bca47bdebfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:16.691242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43edcd-1698-4668-9476-bca47bdebfbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:16.760099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43edcd-1698-4668-9476-bca47bdebfbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:16.761554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43edcd-1698-4668-9476-bca47bdebfbd None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:16.784463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:16.858274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 405/819] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:13:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 19 02:13:16.864474 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a08e4b-333b-4db0-a9b6-c86f521fbc88 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:16.960688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a08e4b-333b-4db0-a9b6-c86f521fbc88 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:16.993765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ff0bb-c63b-4646-b999-47a879bfd914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:16.994406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ff0bb-c63b-4646-b999-47a879bfd914 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:16.994681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ff0bb-c63b-4646-b999-47a879bfd914 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:16.994947 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ff0bb-c63b-4646-b999-47a879bfd914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:17.004991 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ff0bb-c63b-4646-b999-47a879bfd914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:17.006435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ff0bb-c63b-4646-b999-47a879bfd914 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-ab8ff0bb-c63b-4646-b999-47a879bfd914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:13:17.020140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:17.021171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:13:17.022036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:13:17.022036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:13:17.022036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:17.022036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 415/820] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:17.027040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43edcd-1698-4668-9476-bca47bdebfbd None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:17.125149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43edcd-1698-4668-9476-bca47bdebfbd None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:17.147868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:17.158554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769ff1d-de16-4cf4-980e-b2c54cf9cd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:17.165483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769ff1d-de16-4cf4-980e-b2c54cf9cd47 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:17.165483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769ff1d-de16-4cf4-980e-b2c54cf9cd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:17.165483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769ff1d-de16-4cf4-980e-b2c54cf9cd47 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:17.165483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769ff1d-de16-4cf4-980e-b2c54cf9cd47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:17.165483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769ff1d-de16-4cf4-980e-b2c54cf9cd47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:17.165483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769ff1d-de16-4cf4-980e-b2c54cf9cd47 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:17.173245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 406/821] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:17.179174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ff0bb-c63b-4646-b999-47a879bfd914 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:17.227410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:17.276544 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ff0bb-c63b-4646-b999-47a879bfd914 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:17.309352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdb12e-86e1-4e09-95ce-a4e2020053c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:17.309970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdb12e-86e1-4e09-95ce-a4e2020053c3 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:17.310289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdb12e-86e1-4e09-95ce-a4e2020053c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:17.310559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cdb12e-86e1-4e09-95ce-a4e2020053c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:17.318087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdb12e-86e1-4e09-95ce-a4e2020053c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:17.319525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cdb12e-86e1-4e09-95ce-a4e2020053c3 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-d7cdb12e-86e1-4e09-95ce-a4e2020053c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:13:17.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:17.332002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:17.332876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:13:17.332876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:13:17.332876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:13:17.332876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:13:17.332876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:13:17.332876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:13:17.332876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:13:17.332876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:17.332876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 416/822] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:13:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:17.337826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769ff1d-de16-4cf4-980e-b2c54cf9cd47 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:17.434493 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769ff1d-de16-4cf4-980e-b2c54cf9cd47 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:17.468986 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8837e1-85d5-4b46-94bf-13f015115150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:17.469604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8837e1-85d5-4b46-94bf-13f015115150 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:17.469883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8837e1-85d5-4b46-94bf-13f015115150 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:17.470192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8837e1-85d5-4b46-94bf-13f015115150 None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:17.470891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8837e1-85d5-4b46-94bf-13f015115150 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:17.471196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8837e1-85d5-4b46-94bf-13f015115150 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:17.472614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8837e1-85d5-4b46-94bf-13f015115150 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:17.485388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 407/823] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:13:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:17.491148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdb12e-86e1-4e09-95ce-a4e2020053c3 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:17.585987 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cdb12e-86e1-4e09-95ce-a4e2020053c3 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:17.595177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:17.618678 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d2c01-f9af-459b-840d-605bf58415df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:17.619261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d2c01-f9af-459b-840d-605bf58415df None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:17.619538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d2c01-f9af-459b-840d-605bf58415df None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:17.619806 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d2c01-f9af-459b-840d-605bf58415df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:17.620514 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d2c01-f9af-459b-840d-605bf58415df None admin] RBAC: Adding query filter params (role.id=e95a043c68e440b2a961391d6e6e625b, scope.project.id=676e986ff972410d9f498800d313476f, user.id=0abf52399a344157ac4e0f9a8212164e) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:17.620819 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d2c01-f9af-459b-840d-605bf58415df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:17.622272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d2c01-f9af-459b-840d-605bf58415df None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:17.627838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 417/824] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:13:17 2022] GET /identity/v3/role_assignments?user.id=0abf52399a344157ac4e0f9a8212164e&scope.project.id=676e986ff972410d9f498800d313476f&role.id=e95a043c68e440b2a961391d6e6e625b => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:18.718798 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e41658-6939-4123-8878-830e40ae0717 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:18.719351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e41658-6939-4123-8878-830e40ae0717 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:18.719351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e41658-6939-4123-8878-830e40ae0717 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:18.720651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 408/825] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:18.725866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c93f1-c520-4624-aa5d-46a629122fba None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:18.726272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c93f1-c520-4624-aa5d-46a629122fba None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:18.726444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c93f1-c520-4624-aa5d-46a629122fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:18.744526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-6f8c93f1-c520-4624-aa5d-46a629122fba None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:18.745686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6f8c93f1-c520-4624-aa5d-46a629122fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:18.808372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 418/826] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:19.075473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e41658-6939-4123-8878-830e40ae0717 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:19.122646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:19.174642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e41658-6939-4123-8878-830e40ae0717 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:19.206815 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72317bac-8b13-47c0-aa6f-7551db500ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:19.207450 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72317bac-8b13-47c0-aa6f-7551db500ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:19.207706 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72317bac-8b13-47c0-aa6f-7551db500ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:19.207971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72317bac-8b13-47c0-aa6f-7551db500ab7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:19.217740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72317bac-8b13-47c0-aa6f-7551db500ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-72317bac-8b13-47c0-aa6f-7551db500ab7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:13:19.223013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:19.224107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:19.225096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 409/827] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:13:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:19.230057 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c93f1-c520-4624-aa5d-46a629122fba None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:19.322686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c93f1-c520-4624-aa5d-46a629122fba None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:19.324129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:19.356326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e51662-0a46-47ac-aa35-36fd5243ce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:19.357037 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e51662-0a46-47ac-aa35-36fd5243ce53 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:19.357488 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e51662-0a46-47ac-aa35-36fd5243ce53 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:19.357798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e51662-0a46-47ac-aa35-36fd5243ce53 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:19.359279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e51662-0a46-47ac-aa35-36fd5243ce53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:19.359585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e51662-0a46-47ac-aa35-36fd5243ce53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:19.360944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e51662-0a46-47ac-aa35-36fd5243ce53 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:19.368702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 419/828] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:13:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:19.374416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72317bac-8b13-47c0-aa6f-7551db500ab7 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:19.469325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72317bac-8b13-47c0-aa6f-7551db500ab7 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:19.471565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:19.501673 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a46e3-c924-48a2-83e4-2842c53c96d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:19.502302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a46e3-c924-48a2-83e4-2842c53c96d5 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:19.502579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a46e3-c924-48a2-83e4-2842c53c96d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:19.502848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a46e3-c924-48a2-83e4-2842c53c96d5 None admin] PATH_INFO: `/v3/users/65f145edafe842ac857c70af4c0ece07` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:19.515649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a46e3-c924-48a2-83e4-2842c53c96d5 None admin] RBAC: Authorizing `identity:get_user(user_id=65f145edafe842ac857c70af4c0ece07)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:19.517073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a46e3-c924-48a2-83e4-2842c53c96d5 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:19.518644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 410/829] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:13:19 2022] GET /identity/v3/users/65f145edafe842ac857c70af4c0ece07 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:19.524438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e51662-0a46-47ac-aa35-36fd5243ce53 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:19.619526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e51662-0a46-47ac-aa35-36fd5243ce53 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:19.651152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9688b7-ec8f-437b-abe2-9c64988cfb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:19.651759 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9688b7-ec8f-437b-abe2-9c64988cfb47 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:19.652034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9688b7-ec8f-437b-abe2-9c64988cfb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:19.652301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9688b7-ec8f-437b-abe2-9c64988cfb47 None admin] PATH_INFO: `/v3/projects/e69778c6a28f4420b056476980f8d984` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:19.653396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9688b7-ec8f-437b-abe2-9c64988cfb47 None admin] RBAC: Authorizing `identity:get_project(project_id=e69778c6a28f4420b056476980f8d984)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:19.654830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9688b7-ec8f-437b-abe2-9c64988cfb47 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:19.657046 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 420/830] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:13:19 2022] GET /identity/v3/projects/e69778c6a28f4420b056476980f8d984 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:19.662837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a46e3-c924-48a2-83e4-2842c53c96d5 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:19.760543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a46e3-c924-48a2-83e4-2842c53c96d5 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:19.792672 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3e3b0-90c2-42b3-aa6e-d340ea218e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:19.793309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3e3b0-90c2-42b3-aa6e-d340ea218e25 None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:19.793586 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3e3b0-90c2-42b3-aa6e-d340ea218e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:19.793858 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3e3b0-90c2-42b3-aa6e-d340ea218e25 None admin] PATH_INFO: `/v3/projects/e69778c6a28f4420b056476980f8d984/users/65f145edafe842ac857c70af4c0ece07/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:19.795755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3e3b0-90c2-42b3-aa6e-d340ea218e25 None admin] RBAC: Authorizing `identity:create_grant(project_id=e69778c6a28f4420b056476980f8d984, user_id=65f145edafe842ac857c70af4c0ece07, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:19.797831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3e3b0-90c2-42b3-aa6e-d340ea218e25 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:19.884646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 411/831] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:13:19 2022] PUT /identity/v3/projects/e69778c6a28f4420b056476980f8d984/users/65f145edafe842ac857c70af4c0ece07/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:13:21.976237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f214e3-646e-4294-8433-68cd0ce6b19f None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:21.976768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f214e3-646e-4294-8433-68cd0ce6b19f None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:21.976768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f214e3-646e-4294-8433-68cd0ce6b19f None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:21.978113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 421/832] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:21.983416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf8f8f-b9a4-4c08-abf7-7d943694500a None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:21.983867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf8f8f-b9a4-4c08-abf7-7d943694500a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:21.984033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf8f8f-b9a4-4c08-abf7-7d943694500a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:22.005651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ccdf8f8f-b9a4-4c08-abf7-7d943694500a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:22.006884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ccdf8f8f-b9a4-4c08-abf7-7d943694500a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:22.023905 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:22.072179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 412/833] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:22.363396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f214e3-646e-4294-8433-68cd0ce6b19f None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:22.397685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:22.470063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f214e3-646e-4294-8433-68cd0ce6b19f None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:22.503288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaebaf-1765-4dae-a43f-7bed71e837ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:22.503900 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaebaf-1765-4dae-a43f-7bed71e837ed None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:22.504172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaebaf-1765-4dae-a43f-7bed71e837ed None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:22.504436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaebaf-1765-4dae-a43f-7bed71e837ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:22.514303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daaebaf-1765-4dae-a43f-7bed71e837ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-5daaebaf-1765-4dae-a43f-7bed71e837ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:13:22.519590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:22.520741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:22.521754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 422/834] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:13:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:22.527044 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf8f8f-b9a4-4c08-abf7-7d943694500a None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:22.625242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf8f8f-b9a4-4c08-abf7-7d943694500a None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:22.658658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99477e02-18ae-4463-a1e8-0455d41364d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:22.659258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99477e02-18ae-4463-a1e8-0455d41364d1 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:22.659547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99477e02-18ae-4463-a1e8-0455d41364d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:22.659813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99477e02-18ae-4463-a1e8-0455d41364d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:22.660484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99477e02-18ae-4463-a1e8-0455d41364d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:22.660785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99477e02-18ae-4463-a1e8-0455d41364d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:22.662155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99477e02-18ae-4463-a1e8-0455d41364d1 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:22.670427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 413/835] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:13:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:22.676829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaebaf-1765-4dae-a43f-7bed71e837ed None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:22.771526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:22.779704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaebaf-1765-4dae-a43f-7bed71e837ed None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:22.812760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7aa117-8877-4a05-af75-5d572367c188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:22.813366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7aa117-8877-4a05-af75-5d572367c188 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:22.813642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7aa117-8877-4a05-af75-5d572367c188 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:22.813909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7aa117-8877-4a05-af75-5d572367c188 None admin] PATH_INFO: `/v3/users/65f145edafe842ac857c70af4c0ece07` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:22.815050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7aa117-8877-4a05-af75-5d572367c188 None admin] RBAC: Authorizing `identity:get_user(user_id=65f145edafe842ac857c70af4c0ece07)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:22.816463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7aa117-8877-4a05-af75-5d572367c188 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:22.818003 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 423/836] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:13:22 2022] GET /identity/v3/users/65f145edafe842ac857c70af4c0ece07 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:22.823828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99477e02-18ae-4463-a1e8-0455d41364d1 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:22.839355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:22.919391 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99477e02-18ae-4463-a1e8-0455d41364d1 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:22.950827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e8a0a-7962-4e55-8c11-5b0a2877311b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:22.951423 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e8a0a-7962-4e55-8c11-5b0a2877311b None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:22.951697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e8a0a-7962-4e55-8c11-5b0a2877311b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:22.951962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e8a0a-7962-4e55-8c11-5b0a2877311b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:23.015127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e8a0a-7962-4e55-8c11-5b0a2877311b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:23.016547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e8a0a-7962-4e55-8c11-5b0a2877311b None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:23.029901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:23.111141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 414/837] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:13:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 02:13:23.116918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7aa117-8877-4a05-af75-5d572367c188 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:23.210832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7aa117-8877-4a05-af75-5d572367c188 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:23.243412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ceb18-7317-4359-82b5-5655eb4b2260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:23.244009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ceb18-7317-4359-82b5-5655eb4b2260 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:23.244281 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ceb18-7317-4359-82b5-5655eb4b2260 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:23.244547 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ceb18-7317-4359-82b5-5655eb4b2260 None admin] PATH_INFO: `/v3/projects/e69778c6a28f4420b056476980f8d984` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:23.245630 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ceb18-7317-4359-82b5-5655eb4b2260 None admin] RBAC: Authorizing `identity:get_project(project_id=e69778c6a28f4420b056476980f8d984)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:23.247064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ceb18-7317-4359-82b5-5655eb4b2260 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:23.248627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 424/838] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:13:23 2022] GET /identity/v3/projects/e69778c6a28f4420b056476980f8d984 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:23.254552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e8a0a-7962-4e55-8c11-5b0a2877311b None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:23.348358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e8a0a-7962-4e55-8c11-5b0a2877311b None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:23.362130 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:23.380464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de0c9d-ed4f-4488-bbce-21242395534f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:23.381052 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de0c9d-ed4f-4488-bbce-21242395534f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:23.381334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de0c9d-ed4f-4488-bbce-21242395534f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:23.381602 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de0c9d-ed4f-4488-bbce-21242395534f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:23.382344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de0c9d-ed4f-4488-bbce-21242395534f None admin] RBAC: Adding query filter params (role.id=d5070bb151d343ca8de32abe373f4a73, scope.project.id=e69778c6a28f4420b056476980f8d984, user.id=65f145edafe842ac857c70af4c0ece07) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:23.382649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de0c9d-ed4f-4488-bbce-21242395534f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:23.384061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de0c9d-ed4f-4488-bbce-21242395534f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:23.389648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 415/839] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:13:23 2022] GET /identity/v3/role_assignments?user.id=65f145edafe842ac857c70af4c0ece07&scope.project.id=e69778c6a28f4420b056476980f8d984&role.id=d5070bb151d343ca8de32abe373f4a73 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:25.498122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d1582-f53a-4b15-8b3f-7be06bba1a5b None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:25.498626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d1582-f53a-4b15-8b3f-7be06bba1a5b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:25.498713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d1582-f53a-4b15-8b3f-7be06bba1a5b None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:25.500494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 425/840] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:25.505590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a1193-1f1d-47cd-b49b-45e2684fa847 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:25.505975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a1193-1f1d-47cd-b49b-45e2684fa847 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:25.506186 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a1193-1f1d-47cd-b49b-45e2684fa847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:25.524319 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7f0a1193-1f1d-47cd-b49b-45e2684fa847 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:25.525487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7f0a1193-1f1d-47cd-b49b-45e2684fa847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:25.588708 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 416/841] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:13:25.854676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d1582-f53a-4b15-8b3f-7be06bba1a5b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:25.882766 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:25.952856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d1582-f53a-4b15-8b3f-7be06bba1a5b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:25.985778 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b4908-dd51-40e1-87b4-1ce061252380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:25.986409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b4908-dd51-40e1-87b4-1ce061252380 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:25.986686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b4908-dd51-40e1-87b4-1ce061252380 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:25.986958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b4908-dd51-40e1-87b4-1ce061252380 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:25.988069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b4908-dd51-40e1-87b4-1ce061252380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:25.989491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b4908-dd51-40e1-87b4-1ce061252380 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:25.991042 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 426/842] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:25.996740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a1193-1f1d-47cd-b49b-45e2684fa847 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:26.089096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a1193-1f1d-47cd-b49b-45e2684fa847 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:26.098718 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:26.122613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfc392-e661-468c-a6db-b8b10c59ca42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:26.123158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfc392-e661-468c-a6db-b8b10c59ca42 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:26.123434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfc392-e661-468c-a6db-b8b10c59ca42 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:26.123702 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfc392-e661-468c-a6db-b8b10c59ca42 None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:26.124371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfc392-e661-468c-a6db-b8b10c59ca42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:26.125774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfc392-e661-468c-a6db-b8b10c59ca42 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:26.205476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 417/843] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:13:25 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:13:28.353832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2f128-87f9-41dc-b644-8bc977ac9c04 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:28.354430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2f128-87f9-41dc-b644-8bc977ac9c04 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:28.354430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2f128-87f9-41dc-b644-8bc977ac9c04 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:28.355725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 427/844] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:28.360815 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091229e6-80d6-44a8-8804-a8f2a6d06053 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:28.361221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091229e6-80d6-44a8-8804-a8f2a6d06053 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:28.361364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091229e6-80d6-44a8-8804-a8f2a6d06053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:28.379519 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-091229e6-80d6-44a8-8804-a8f2a6d06053 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:28.380684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-091229e6-80d6-44a8-8804-a8f2a6d06053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:28.443199 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 418/845] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:28.712050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2f128-87f9-41dc-b644-8bc977ac9c04 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:28.804810 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:28.811157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2f128-87f9-41dc-b644-8bc977ac9c04 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:28.844480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8622fd0-71b0-4910-9faf-83a636e1e7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:28.845425 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8622fd0-71b0-4910-9faf-83a636e1e7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:28.845855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8622fd0-71b0-4910-9faf-83a636e1e7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:28.846459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8622fd0-71b0-4910-9faf-83a636e1e7e7 None admin] PATH_INFO: `/v3/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:28.847877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8622fd0-71b0-4910-9faf-83a636e1e7e7 None admin] RBAC: Authorizing `identity:get_role(role_id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:28.849233 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8622fd0-71b0-4910-9faf-83a636e1e7e7 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:28.850717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 428/846] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:13:28 2022] GET /identity/v3/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:28.856531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091229e6-80d6-44a8-8804-a8f2a6d06053 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:28.944302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:28.950619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091229e6-80d6-44a8-8804-a8f2a6d06053 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:28.983259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc2b39-8f50-4be8-bfdc-577ff4465654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:28.983842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc2b39-8f50-4be8-bfdc-577ff4465654 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:28.984119 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc2b39-8f50-4be8-bfdc-577ff4465654 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:28.984388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc2b39-8f50-4be8-bfdc-577ff4465654 None admin] PATH_INFO: `/v3/users/4c72f216b0b3429d86d0dc3b05624567` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:28.996687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc2b39-8f50-4be8-bfdc-577ff4465654 None admin] RBAC: Authorizing `identity:get_user(user_id=4c72f216b0b3429d86d0dc3b05624567)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:28.998131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc2b39-8f50-4be8-bfdc-577ff4465654 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:29.000209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 419/847] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:13:28 2022] GET /identity/v3/users/4c72f216b0b3429d86d0dc3b05624567 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:29.005823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8622fd0-71b0-4910-9faf-83a636e1e7e7 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:29.098775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8622fd0-71b0-4910-9faf-83a636e1e7e7 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:29.131853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f7cfe-3bd2-45a8-a99c-0b0273fed9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:29.132451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f7cfe-3bd2-45a8-a99c-0b0273fed9ed None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:29.132723 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f7cfe-3bd2-45a8-a99c-0b0273fed9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:29.132985 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f7cfe-3bd2-45a8-a99c-0b0273fed9ed None admin] PATH_INFO: `/v3/projects/e69778c6a28f4420b056476980f8d984` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:29.134078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f7cfe-3bd2-45a8-a99c-0b0273fed9ed None admin] RBAC: Authorizing `identity:get_project(project_id=e69778c6a28f4420b056476980f8d984)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:29.135478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f7cfe-3bd2-45a8-a99c-0b0273fed9ed None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:29.137059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 429/848] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:13:29 2022] GET /identity/v3/projects/e69778c6a28f4420b056476980f8d984 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:29.143562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc2b39-8f50-4be8-bfdc-577ff4465654 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:29.237960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc2b39-8f50-4be8-bfdc-577ff4465654 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:29.270595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81240ebe-5b32-49b7-8eb7-eda6bf8ab5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:29.271237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81240ebe-5b32-49b7-8eb7-eda6bf8ab5f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:29.271514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81240ebe-5b32-49b7-8eb7-eda6bf8ab5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:29.271783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81240ebe-5b32-49b7-8eb7-eda6bf8ab5f0 None admin] PATH_INFO: `/v3/projects/e69778c6a28f4420b056476980f8d984/users/4c72f216b0b3429d86d0dc3b05624567/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:29.273886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81240ebe-5b32-49b7-8eb7-eda6bf8ab5f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e69778c6a28f4420b056476980f8d984, user_id=4c72f216b0b3429d86d0dc3b05624567, role_id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:29.275322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81240ebe-5b32-49b7-8eb7-eda6bf8ab5f0 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:29.359850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 420/849] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:13:29 2022] PUT /identity/v3/projects/e69778c6a28f4420b056476980f8d984/users/4c72f216b0b3429d86d0dc3b05624567/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:13:31.463172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd17a0-7738-436e-beeb-dea90ff80a36 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:31.463664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd17a0-7738-436e-beeb-dea90ff80a36 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:31.463664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd17a0-7738-436e-beeb-dea90ff80a36 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:31.464954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 430/850] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:31.470950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b39368-7181-4da4-915c-17fb9e5dcf29 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:31.471339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b39368-7181-4da4-915c-17fb9e5dcf29 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:31.471494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b39368-7181-4da4-915c-17fb9e5dcf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:31.490213 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-40b39368-7181-4da4-915c-17fb9e5dcf29 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:31.491370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-40b39368-7181-4da4-915c-17fb9e5dcf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:31.498248 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:31.667187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 421/851] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:31.938915 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd17a0-7738-436e-beeb-dea90ff80a36 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:31.969379 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:32.036797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd17a0-7738-436e-beeb-dea90ff80a36 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:32.068770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4543eb-19ff-48c0-aac4-b98e60b3c038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:32.069375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4543eb-19ff-48c0-aac4-b98e60b3c038 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:32.069651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4543eb-19ff-48c0-aac4-b98e60b3c038 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:32.069921 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4543eb-19ff-48c0-aac4-b98e60b3c038 None admin] PATH_INFO: `/v3/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:32.071306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4543eb-19ff-48c0-aac4-b98e60b3c038 None admin] RBAC: Authorizing `identity:get_role(role_id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:32.072655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4543eb-19ff-48c0-aac4-b98e60b3c038 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:32.073932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 431/852] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:13:31 2022] GET /identity/v3/roles/f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:32.080434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b39368-7181-4da4-915c-17fb9e5dcf29 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:32.173552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b39368-7181-4da4-915c-17fb9e5dcf29 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:32.205586 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180767c-b4fd-4bdc-bc96-c60f6c140c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:32.206240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180767c-b4fd-4bdc-bc96-c60f6c140c0a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:32.206514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180767c-b4fd-4bdc-bc96-c60f6c140c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:32.206781 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180767c-b4fd-4bdc-bc96-c60f6c140c0a None admin] PATH_INFO: `/v3/users/4c72f216b0b3429d86d0dc3b05624567` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:32.207872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180767c-b4fd-4bdc-bc96-c60f6c140c0a None admin] RBAC: Authorizing `identity:get_user(user_id=4c72f216b0b3429d86d0dc3b05624567)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:32.209353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180767c-b4fd-4bdc-bc96-c60f6c140c0a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:32.210896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 422/853] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:13:32 2022] GET /identity/v3/users/4c72f216b0b3429d86d0dc3b05624567 => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:32.217011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4543eb-19ff-48c0-aac4-b98e60b3c038 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:32.310146 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:32.313512 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4543eb-19ff-48c0-aac4-b98e60b3c038 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:32.345224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5751b6e-c434-42a4-aaed-b9345c01f48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:32.354784 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5751b6e-c434-42a4-aaed-b9345c01f48e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:32.354784 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5751b6e-c434-42a4-aaed-b9345c01f48e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:32.354784 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5751b6e-c434-42a4-aaed-b9345c01f48e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:32.410129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5751b6e-c434-42a4-aaed-b9345c01f48e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:32.411545 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5751b6e-c434-42a4-aaed-b9345c01f48e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:32.504216 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:32.507720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 432/854] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:13:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 02:13:32.513841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180767c-b4fd-4bdc-bc96-c60f6c140c0a None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:32.525578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:32.609489 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180767c-b4fd-4bdc-bc96-c60f6c140c0a None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:32.641208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee112e7d-ce63-4861-bc03-bf0b33704287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:32.641821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112e7d-ce63-4861-bc03-bf0b33704287 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:32.642127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112e7d-ce63-4861-bc03-bf0b33704287 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:32.642401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112e7d-ce63-4861-bc03-bf0b33704287 None admin] PATH_INFO: `/v3/projects/e69778c6a28f4420b056476980f8d984` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:32.643495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee112e7d-ce63-4861-bc03-bf0b33704287 None admin] RBAC: Authorizing `identity:get_project(project_id=e69778c6a28f4420b056476980f8d984)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:32.644888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee112e7d-ce63-4861-bc03-bf0b33704287 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:32.646503 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 423/855] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:13:32 2022] GET /identity/v3/projects/e69778c6a28f4420b056476980f8d984 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:32.652751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5751b6e-c434-42a4-aaed-b9345c01f48e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:32.746147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5751b6e-c434-42a4-aaed-b9345c01f48e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:32.778261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599d264-78d3-413e-91f7-90aa84a1ed68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:32.778786 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599d264-78d3-413e-91f7-90aa84a1ed68 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:32.779058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599d264-78d3-413e-91f7-90aa84a1ed68 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:32.779321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599d264-78d3-413e-91f7-90aa84a1ed68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:32.780025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599d264-78d3-413e-91f7-90aa84a1ed68 None admin] RBAC: Adding query filter params (role.id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6, scope.project.id=e69778c6a28f4420b056476980f8d984, user.id=4c72f216b0b3429d86d0dc3b05624567) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:32.780325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599d264-78d3-413e-91f7-90aa84a1ed68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:32.782271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599d264-78d3-413e-91f7-90aa84a1ed68 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:32.787724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 433/856] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:13:32 2022] GET /identity/v3/role_assignments?user.id=4c72f216b0b3429d86d0dc3b05624567&scope.project.id=e69778c6a28f4420b056476980f8d984&role.id=f2f3bbd00d6f4bbcb036bfa5e2d8c7e6 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:34.885879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693992b-9148-456b-92d1-9f1a87443990 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:34.886492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693992b-9148-456b-92d1-9f1a87443990 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:34.886492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693992b-9148-456b-92d1-9f1a87443990 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:34.887768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 424/857] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:34.893354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09c10b-757d-4d08-b9b8-016cbb5dd241 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:34.893731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09c10b-757d-4d08-b9b8-016cbb5dd241 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:34.893892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09c10b-757d-4d08-b9b8-016cbb5dd241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:34.911910 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-3b09c10b-757d-4d08-b9b8-016cbb5dd241 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:34.913052 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3b09c10b-757d-4d08-b9b8-016cbb5dd241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:34.968703 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:34.976380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 434/858] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:35.239099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693992b-9148-456b-92d1-9f1a87443990 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:35.314956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:35.335400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693992b-9148-456b-92d1-9f1a87443990 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:35.367681 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324eb73-9aed-4f3f-a311-ad4e4c94e340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:35.368285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324eb73-9aed-4f3f-a311-ad4e4c94e340 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:35.368556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324eb73-9aed-4f3f-a311-ad4e4c94e340 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:35.368822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324eb73-9aed-4f3f-a311-ad4e4c94e340 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:35.369895 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324eb73-9aed-4f3f-a311-ad4e4c94e340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:35.371320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324eb73-9aed-4f3f-a311-ad4e4c94e340 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:35.372821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 425/859] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:35.379105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09c10b-757d-4d08-b9b8-016cbb5dd241 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:35.471058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09c10b-757d-4d08-b9b8-016cbb5dd241 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:35.503011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc3f00-494b-42b3-a335-2bcd45681130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:35.503539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc3f00-494b-42b3-a335-2bcd45681130 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:35.503812 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc3f00-494b-42b3-a335-2bcd45681130 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:35.504078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc3f00-494b-42b3-a335-2bcd45681130 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:35.504710 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc3f00-494b-42b3-a335-2bcd45681130 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:35.506132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc3f00-494b-42b3-a335-2bcd45681130 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:35.586504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 435/860] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:13:35 2022] POST /identity/v3/projects => generated 311 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:13:37.727525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd07b6-e7a1-4c73-9de2-81662feb8ee1 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:37.728082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd07b6-e7a1-4c73-9de2-81662feb8ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:37.728082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd07b6-e7a1-4c73-9de2-81662feb8ee1 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:37.729381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 426/861] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:37.734456 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea981e7-8ece-460c-826d-89c9756a0216 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:37.734885 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea981e7-8ece-460c-826d-89c9756a0216 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:37.734989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea981e7-8ece-460c-826d-89c9756a0216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:37.752593 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-1ea981e7-8ece-460c-826d-89c9756a0216 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:37.753761 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1ea981e7-8ece-460c-826d-89c9756a0216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:37.794247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:37.817255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 436/862] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:38.083696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd07b6-e7a1-4c73-9de2-81662feb8ee1 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:38.181275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd07b6-e7a1-4c73-9de2-81662feb8ee1 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:38.212903 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5373cf7-77aa-4525-814b-cfffc55a40d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:38.213512 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5373cf7-77aa-4525-814b-cfffc55a40d9 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:38.213787 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5373cf7-77aa-4525-814b-cfffc55a40d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:38.214069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5373cf7-77aa-4525-814b-cfffc55a40d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:38.215175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5373cf7-77aa-4525-814b-cfffc55a40d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:38.216843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5373cf7-77aa-4525-814b-cfffc55a40d9 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:38.218912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 427/863] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:13:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:38.224553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea981e7-8ece-460c-826d-89c9756a0216 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:38.318441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea981e7-8ece-460c-826d-89c9756a0216 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:38.350244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba581ec-51d0-4a8d-b732-c1776a63a448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:38.350775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba581ec-51d0-4a8d-b732-c1776a63a448 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:38.351052 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba581ec-51d0-4a8d-b732-c1776a63a448 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:38.351321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba581ec-51d0-4a8d-b732-c1776a63a448 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:38.351971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba581ec-51d0-4a8d-b732-c1776a63a448 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:38.353381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba581ec-51d0-4a8d-b732-c1776a63a448 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:38.442146 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 437/864] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:13:38 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:13:40.612017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60d3dc-fd9b-4a62-8d4b-26abfdeab1d6 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:40.612539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60d3dc-fd9b-4a62-8d4b-26abfdeab1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:40.612539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60d3dc-fd9b-4a62-8d4b-26abfdeab1d6 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:40.613873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 428/865] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:40.619040 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85166a3-4bdd-4892-a05e-665f98a688be None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:40.619458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85166a3-4bdd-4892-a05e-665f98a688be None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:40.619584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85166a3-4bdd-4892-a05e-665f98a688be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:40.637005 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b85166a3-4bdd-4892-a05e-665f98a688be None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:40.638192 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b85166a3-4bdd-4892-a05e-665f98a688be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:40.663248 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:40.701823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 438/866] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:40.977649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60d3dc-fd9b-4a62-8d4b-26abfdeab1d6 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:40.993375 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:41.075112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60d3dc-fd9b-4a62-8d4b-26abfdeab1d6 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:41.107997 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2db198-fc52-47cd-b599-710ed023864f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:41.108622 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2db198-fc52-47cd-b599-710ed023864f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:41.108899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2db198-fc52-47cd-b599-710ed023864f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:41.109730 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2db198-fc52-47cd-b599-710ed023864f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:41.119529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2db198-fc52-47cd-b599-710ed023864f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-bb2db198-fc52-47cd-b599-710ed023864f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:13:41.124259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:41.125371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:41.126394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 429/867] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:13:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:41.131311 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85166a3-4bdd-4892-a05e-665f98a688be None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:41.226220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85166a3-4bdd-4892-a05e-665f98a688be None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:41.258563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e4d9f-b0dc-4bc5-8e3a-f0959ddb0c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:41.259180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e4d9f-b0dc-4bc5-8e3a-f0959ddb0c5b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:41.259459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e4d9f-b0dc-4bc5-8e3a-f0959ddb0c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:41.259729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e4d9f-b0dc-4bc5-8e3a-f0959ddb0c5b None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:41.260380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e4d9f-b0dc-4bc5-8e3a-f0959ddb0c5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:41.260681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e4d9f-b0dc-4bc5-8e3a-f0959ddb0c5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:41.262057 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e4d9f-b0dc-4bc5-8e3a-f0959ddb0c5b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:41.269851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 439/868] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:41.275411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2db198-fc52-47cd-b599-710ed023864f None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:41.348467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:41.371457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2db198-fc52-47cd-b599-710ed023864f None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:41.404123 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca0b1f-f670-42d8-b190-5e87618825ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:41.404729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0b1f-f670-42d8-b190-5e87618825ee None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:41.405004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0b1f-f670-42d8-b190-5e87618825ee None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:41.405272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0b1f-f670-42d8-b190-5e87618825ee None admin] PATH_INFO: `/v3/users/8e30d402224f4556bac59e79f915fa0a` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:41.417605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0b1f-f670-42d8-b190-5e87618825ee None admin] RBAC: Authorizing `identity:get_user(user_id=8e30d402224f4556bac59e79f915fa0a)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:41.419049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0b1f-f670-42d8-b190-5e87618825ee None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:41.420582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 430/869] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:13:41 2022] GET /identity/v3/users/8e30d402224f4556bac59e79f915fa0a => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:41.426736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e4d9f-b0dc-4bc5-8e3a-f0959ddb0c5b None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:41.451002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:41.522718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e4d9f-b0dc-4bc5-8e3a-f0959ddb0c5b None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:41.554795 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252ab04-a030-488e-9923-affdc5193599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:41.555398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252ab04-a030-488e-9923-affdc5193599 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:41.555673 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252ab04-a030-488e-9923-affdc5193599 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:41.555941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252ab04-a030-488e-9923-affdc5193599 None admin] PATH_INFO: `/v3/projects/b07aac10f56f4c2a9e1fb5cc41e7336c` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:41.557036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252ab04-a030-488e-9923-affdc5193599 None admin] RBAC: Authorizing `identity:get_project(project_id=b07aac10f56f4c2a9e1fb5cc41e7336c)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:41.558469 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252ab04-a030-488e-9923-affdc5193599 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:41.560042 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 440/870] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:13:41 2022] GET /identity/v3/projects/b07aac10f56f4c2a9e1fb5cc41e7336c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:41.566509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca0b1f-f670-42d8-b190-5e87618825ee None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:41.660374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca0b1f-f670-42d8-b190-5e87618825ee None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:41.688325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:41.693069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd978f-8950-4573-b3a9-3b485b3a017f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:41.693721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd978f-8950-4573-b3a9-3b485b3a017f None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:41.693998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd978f-8950-4573-b3a9-3b485b3a017f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:41.694305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd978f-8950-4573-b3a9-3b485b3a017f None admin] PATH_INFO: `/v3/projects/b07aac10f56f4c2a9e1fb5cc41e7336c/users/8e30d402224f4556bac59e79f915fa0a/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:41.696227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd978f-8950-4573-b3a9-3b485b3a017f None admin] RBAC: Authorizing `identity:create_grant(project_id=b07aac10f56f4c2a9e1fb5cc41e7336c, user_id=8e30d402224f4556bac59e79f915fa0a, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:41.697635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd978f-8950-4573-b3a9-3b485b3a017f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:41.782822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 431/871] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:13:41 2022] PUT /identity/v3/projects/b07aac10f56f4c2a9e1fb5cc41e7336c/users/8e30d402224f4556bac59e79f915fa0a/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:13:43.863773 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb8de-8e99-4def-844d-3cdf4614da0d None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:43.864295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb8de-8e99-4def-844d-3cdf4614da0d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:43.864295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb8de-8e99-4def-844d-3cdf4614da0d None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:43.865622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 441/872] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:43.870783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33105073-6b4d-401e-8e6a-280d9d78420c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:43.871187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33105073-6b4d-401e-8e6a-280d9d78420c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:43.871329 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33105073-6b4d-401e-8e6a-280d9d78420c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:43.889519 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-33105073-6b4d-401e-8e6a-280d9d78420c None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:43.890771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-33105073-6b4d-401e-8e6a-280d9d78420c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:43.953498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 432/873] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:44.218116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccb8de-8e99-4def-844d-3cdf4614da0d None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:44.305660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:44.314597 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccb8de-8e99-4def-844d-3cdf4614da0d None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:44.347701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6995b2-78e3-4d16-845a-4b3d116dbfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:44.348303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6995b2-78e3-4d16-845a-4b3d116dbfdc None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:44.348581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6995b2-78e3-4d16-845a-4b3d116dbfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:44.348849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6995b2-78e3-4d16-845a-4b3d116dbfdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:44.358455 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6995b2-78e3-4d16-845a-4b3d116dbfdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-be6995b2-78e3-4d16-845a-4b3d116dbfdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:13:44.363198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:44.364305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:44.365155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 442/874] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:13:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:44.370214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33105073-6b4d-401e-8e6a-280d9d78420c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:44.462786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33105073-6b4d-401e-8e6a-280d9d78420c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:44.484634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:44.495402 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694caf6-d654-45de-8b37-2b64ff5f9e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:44.495994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694caf6-d654-45de-8b37-2b64ff5f9e29 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:44.496270 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694caf6-d654-45de-8b37-2b64ff5f9e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:44.496539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694caf6-d654-45de-8b37-2b64ff5f9e29 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:44.499006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694caf6-d654-45de-8b37-2b64ff5f9e29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:44.499006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694caf6-d654-45de-8b37-2b64ff5f9e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:44.499006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694caf6-d654-45de-8b37-2b64ff5f9e29 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:44.506465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 433/875] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:13:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:44.512240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6995b2-78e3-4d16-845a-4b3d116dbfdc None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:44.610217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6995b2-78e3-4d16-845a-4b3d116dbfdc None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:44.642356 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0420d1-a832-4f2a-9960-998904375c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:44.642960 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0420d1-a832-4f2a-9960-998904375c99 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:44.643234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0420d1-a832-4f2a-9960-998904375c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:44.643500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0420d1-a832-4f2a-9960-998904375c99 None admin] PATH_INFO: `/v3/users/8e30d402224f4556bac59e79f915fa0a` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:44.644600 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0420d1-a832-4f2a-9960-998904375c99 None admin] RBAC: Authorizing `identity:get_user(user_id=8e30d402224f4556bac59e79f915fa0a)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:44.646119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0420d1-a832-4f2a-9960-998904375c99 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:44.647629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 443/876] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:13:44 2022] GET /identity/v3/users/8e30d402224f4556bac59e79f915fa0a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:44.653507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694caf6-d654-45de-8b37-2b64ff5f9e29 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:44.749353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694caf6-d654-45de-8b37-2b64ff5f9e29 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:44.781936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e3dd5-806b-4bf3-bfc8-e8cb2d8f6e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:44.782555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e3dd5-806b-4bf3-bfc8-e8cb2d8f6e43 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:44.782833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e3dd5-806b-4bf3-bfc8-e8cb2d8f6e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:44.783102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e3dd5-806b-4bf3-bfc8-e8cb2d8f6e43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:44.831694 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:44.846522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e3dd5-806b-4bf3-bfc8-e8cb2d8f6e43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:44.847944 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e3dd5-806b-4bf3-bfc8-e8cb2d8f6e43 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:44.942386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 434/877] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:13:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 02:13:44.948312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0420d1-a832-4f2a-9960-998904375c99 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:44.970033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:45.042515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0420d1-a832-4f2a-9960-998904375c99 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:45.074680 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cecd9-4c81-409b-a351-7be0ed8fa047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:45.075278 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cecd9-4c81-409b-a351-7be0ed8fa047 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:45.075550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cecd9-4c81-409b-a351-7be0ed8fa047 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:45.075822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cecd9-4c81-409b-a351-7be0ed8fa047 None admin] PATH_INFO: `/v3/projects/b07aac10f56f4c2a9e1fb5cc41e7336c` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:45.076907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cecd9-4c81-409b-a351-7be0ed8fa047 None admin] RBAC: Authorizing `identity:get_project(project_id=b07aac10f56f4c2a9e1fb5cc41e7336c)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:45.078376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cecd9-4c81-409b-a351-7be0ed8fa047 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:45.079936 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 444/878] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:13:44 2022] GET /identity/v3/projects/b07aac10f56f4c2a9e1fb5cc41e7336c => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:45.086055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e3dd5-806b-4bf3-bfc8-e8cb2d8f6e43 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:45.165267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:45.180833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e3dd5-806b-4bf3-bfc8-e8cb2d8f6e43 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:45.213250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c8863-2f80-43d3-8774-65cd39ec9834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:45.213836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c8863-2f80-43d3-8774-65cd39ec9834 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:45.214138 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c8863-2f80-43d3-8774-65cd39ec9834 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:45.214407 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c8863-2f80-43d3-8774-65cd39ec9834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:45.215122 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c8863-2f80-43d3-8774-65cd39ec9834 None admin] RBAC: Adding query filter params (role.id=d5070bb151d343ca8de32abe373f4a73, scope.project.id=b07aac10f56f4c2a9e1fb5cc41e7336c, user.id=8e30d402224f4556bac59e79f915fa0a) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:45.215422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c8863-2f80-43d3-8774-65cd39ec9834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:45.217071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c8863-2f80-43d3-8774-65cd39ec9834 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:45.222634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 435/879] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:13:45 2022] GET /identity/v3/role_assignments?user.id=8e30d402224f4556bac59e79f915fa0a&scope.project.id=b07aac10f56f4c2a9e1fb5cc41e7336c&role.id=d5070bb151d343ca8de32abe373f4a73 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:47.357077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cd6ca-3daf-45f3-ae6f-4b15dedebf39 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:47.357637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cd6ca-3daf-45f3-ae6f-4b15dedebf39 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:47.357637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cd6ca-3daf-45f3-ae6f-4b15dedebf39 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:47.358897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 445/880] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:47.364812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1deae-8dca-44d5-b594-cbc49975f770 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:47.365204 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1deae-8dca-44d5-b594-cbc49975f770 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:47.365350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1deae-8dca-44d5-b594-cbc49975f770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:47.383639 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-00b1deae-8dca-44d5-b594-cbc49975f770 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:47.384788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-00b1deae-8dca-44d5-b594-cbc49975f770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:47.446587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 436/881] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:47.711280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cd6ca-3daf-45f3-ae6f-4b15dedebf39 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:47.796333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:47.808782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cd6ca-3daf-45f3-ae6f-4b15dedebf39 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:47.840216 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd37e6-b807-475a-b4fe-7a06a937b94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:47.840806 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd37e6-b807-475a-b4fe-7a06a937b94d None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:47.841077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd37e6-b807-475a-b4fe-7a06a937b94d None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:47.841343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd37e6-b807-475a-b4fe-7a06a937b94d None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:47.842292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd37e6-b807-475a-b4fe-7a06a937b94d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:47.844136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd37e6-b807-475a-b4fe-7a06a937b94d None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:47.920881 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 446/882] 10.222.0.20 () {60 vars in 1138 bytes} [Wed Oct 19 02:13:47 2022] POST /identity/v3/domains => generated 267 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:13:50.082030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db8df6-cd0a-4127-bc92-046eb6da6926 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:50.082590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db8df6-cd0a-4127-bc92-046eb6da6926 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:50.082590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db8df6-cd0a-4127-bc92-046eb6da6926 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:50.083934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 437/883] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:50.089276 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d32c5-8570-48bb-94c1-e4a149fc0003 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:50.089689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d32c5-8570-48bb-94c1-e4a149fc0003 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:50.089795 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d32c5-8570-48bb-94c1-e4a149fc0003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:50.112017 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-096d32c5-8570-48bb-94c1-e4a149fc0003 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:50.113206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-096d32c5-8570-48bb-94c1-e4a149fc0003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:50.176592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 447/884] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:50.448372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db8df6-cd0a-4127-bc92-046eb6da6926 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:50.524523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:50.545783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db8df6-cd0a-4127-bc92-046eb6da6926 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:50.577554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773b2c2-8369-403c-a622-d9fbd025fd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:50.578778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773b2c2-8369-403c-a622-d9fbd025fd03 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:50.579058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773b2c2-8369-403c-a622-d9fbd025fd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:50.579329 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773b2c2-8369-403c-a622-d9fbd025fd03 None admin] PATH_INFO: `/v3/domains/de9ae232406742a4a224687e2a95ff53` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:50.589346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773b2c2-8369-403c-a622-d9fbd025fd03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=de9ae232406742a4a224687e2a95ff53)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:50.590791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773b2c2-8369-403c-a622-d9fbd025fd03 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:50.592596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 438/885] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Oct 19 02:13:50 2022] GET /identity/v3/domains/de9ae232406742a4a224687e2a95ff53 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:50.599117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d32c5-8570-48bb-94c1-e4a149fc0003 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:50.676224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:50.697019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d32c5-8570-48bb-94c1-e4a149fc0003 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:50.730292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18abf1-558c-4952-9d91-0723280bca54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:50.730940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18abf1-558c-4952-9d91-0723280bca54 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:50.731221 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18abf1-558c-4952-9d91-0723280bca54 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:50.731491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18abf1-558c-4952-9d91-0723280bca54 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:50.732131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18abf1-558c-4952-9d91-0723280bca54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:50.733576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18abf1-558c-4952-9d91-0723280bca54 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:50.864670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 448/886] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:13:50 2022] POST /identity/v3/projects => generated 361 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:13:53.010638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef3cc5-f4a2-4409-a4bc-c7e1d6f61dd5 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:53.011190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef3cc5-f4a2-4409-a4bc-c7e1d6f61dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:53.011190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef3cc5-f4a2-4409-a4bc-c7e1d6f61dd5 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:53.012506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 439/887] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:53.018200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73eedd-788d-4866-8aa6-cc0f623e89ef None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:53.018590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73eedd-788d-4866-8aa6-cc0f623e89ef None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:53.018748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73eedd-788d-4866-8aa6-cc0f623e89ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:53.036369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-cc73eedd-788d-4866-8aa6-cc0f623e89ef None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:53.037524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cc73eedd-788d-4866-8aa6-cc0f623e89ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:53.100524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 449/888] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:53.367032 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef3cc5-f4a2-4409-a4bc-c7e1d6f61dd5 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:53.462812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef3cc5-f4a2-4409-a4bc-c7e1d6f61dd5 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:53.494475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a9a38-d8a1-448d-98f2-11274369165a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:53.495084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a9a38-d8a1-448d-98f2-11274369165a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:53.495358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a9a38-d8a1-448d-98f2-11274369165a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:53.495625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a9a38-d8a1-448d-98f2-11274369165a None admin] PATH_INFO: `/v3/domains/de9ae232406742a4a224687e2a95ff53` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:53.496714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a9a38-d8a1-448d-98f2-11274369165a None admin] RBAC: Authorizing `identity:get_domain(domain_id=de9ae232406742a4a224687e2a95ff53)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:53.498140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a9a38-d8a1-448d-98f2-11274369165a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:53.499651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 440/889] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Oct 19 02:13:53 2022] GET /identity/v3/domains/de9ae232406742a4a224687e2a95ff53 => generated 267 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:53.505334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73eedd-788d-4866-8aa6-cc0f623e89ef None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:53.560858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:53.599674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73eedd-788d-4866-8aa6-cc0f623e89ef None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:53.632903 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67924278-b539-4d74-b070-ca4d77cae581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:53.633441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67924278-b539-4d74-b070-ca4d77cae581 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:53.633721 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67924278-b539-4d74-b070-ca4d77cae581 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:53.633992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67924278-b539-4d74-b070-ca4d77cae581 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:53.634670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67924278-b539-4d74-b070-ca4d77cae581 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:53.636088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67924278-b539-4d74-b070-ca4d77cae581 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:53.731721 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 450/890] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:13:53 2022] POST /identity/v3/users => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:13:55.883238 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcfd94-2f31-4172-9e66-f1a400b300db None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:55.883781 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcfd94-2f31-4172-9e66-f1a400b300db None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:55.883781 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bcfd94-2f31-4172-9e66-f1a400b300db None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:55.885121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 441/891] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:55.890098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a43a2-1b40-43f0-a0bb-00540c64282f None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:55.890472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a43a2-1b40-43f0-a0bb-00540c64282f None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:55.891216 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a43a2-1b40-43f0-a0bb-00540c64282f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:55.909285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-bd3a43a2-1b40-43f0-a0bb-00540c64282f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:55.910484 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-bd3a43a2-1b40-43f0-a0bb-00540c64282f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:55.973539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 451/892] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:56.236104 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcfd94-2f31-4172-9e66-f1a400b300db None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:56.245290 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:56.334405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bcfd94-2f31-4172-9e66-f1a400b300db None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:56.366146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae04e2b-8e79-4225-ae28-1718906cface None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:56.366754 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae04e2b-8e79-4225-ae28-1718906cface None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:56.367030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae04e2b-8e79-4225-ae28-1718906cface None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:56.367300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae04e2b-8e79-4225-ae28-1718906cface None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:56.377912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae04e2b-8e79-4225-ae28-1718906cface None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-2ae04e2b-8e79-4225-ae28-1718906cface None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:13:56.382613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:56.383791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:13:56.385033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 442/893] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:13:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:56.389513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a43a2-1b40-43f0-a0bb-00540c64282f None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:56.433446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:56.484443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a43a2-1b40-43f0-a0bb-00540c64282f None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:56.516688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668143a-50da-454e-9f42-8206f49a6256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:56.517269 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668143a-50da-454e-9f42-8206f49a6256 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:56.517542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668143a-50da-454e-9f42-8206f49a6256 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:56.517809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668143a-50da-454e-9f42-8206f49a6256 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:56.518482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668143a-50da-454e-9f42-8206f49a6256 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:56.518777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668143a-50da-454e-9f42-8206f49a6256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:56.520121 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668143a-50da-454e-9f42-8206f49a6256 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:56.527718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 452/894] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:13:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:56.533953 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae04e2b-8e79-4225-ae28-1718906cface None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:56.595999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:56.628370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae04e2b-8e79-4225-ae28-1718906cface None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:56.663403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6513423-af36-4ac1-b5d7-3626babbba4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:56.664040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6513423-af36-4ac1-b5d7-3626babbba4f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:56.664501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6513423-af36-4ac1-b5d7-3626babbba4f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:56.664962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6513423-af36-4ac1-b5d7-3626babbba4f None admin] PATH_INFO: `/v3/users/794de010289243889e61b8c49aa6bc02` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:56.749843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6513423-af36-4ac1-b5d7-3626babbba4f None admin] RBAC: Authorizing `identity:get_user(user_id=794de010289243889e61b8c49aa6bc02)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:56.751844 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6513423-af36-4ac1-b5d7-3626babbba4f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:56.753453 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 443/895] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:13:56 2022] GET /identity/v3/users/794de010289243889e61b8c49aa6bc02 => generated 311 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:56.759234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668143a-50da-454e-9f42-8206f49a6256 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:56.853535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668143a-50da-454e-9f42-8206f49a6256 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:56.861738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:56.885370 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbb5a5-d908-4cc7-9fbf-70f1819ab766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:56.885985 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbb5a5-d908-4cc7-9fbf-70f1819ab766 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:56.886287 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbb5a5-d908-4cc7-9fbf-70f1819ab766 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:56.886559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbb5a5-d908-4cc7-9fbf-70f1819ab766 None admin] PATH_INFO: `/v3/projects/73f224d1387f42d988141cc67de12919` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:56.887673 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbb5a5-d908-4cc7-9fbf-70f1819ab766 None admin] RBAC: Authorizing `identity:get_project(project_id=73f224d1387f42d988141cc67de12919)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:56.889094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbb5a5-d908-4cc7-9fbf-70f1819ab766 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:56.891605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 453/896] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:13:56 2022] GET /identity/v3/projects/73f224d1387f42d988141cc67de12919 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:56.897021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6513423-af36-4ac1-b5d7-3626babbba4f None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:56.990310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6513423-af36-4ac1-b5d7-3626babbba4f None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:57.014112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:57.024599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d962fc8-5a75-405e-8b05-b230ed7f619d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:57.025241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d962fc8-5a75-405e-8b05-b230ed7f619d None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:57.025511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d962fc8-5a75-405e-8b05-b230ed7f619d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:57.025777 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d962fc8-5a75-405e-8b05-b230ed7f619d None admin] PATH_INFO: `/v3/projects/73f224d1387f42d988141cc67de12919/users/794de010289243889e61b8c49aa6bc02/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:57.027644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d962fc8-5a75-405e-8b05-b230ed7f619d None admin] RBAC: Authorizing `identity:create_grant(project_id=73f224d1387f42d988141cc67de12919, user_id=794de010289243889e61b8c49aa6bc02, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:57.029049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d962fc8-5a75-405e-8b05-b230ed7f619d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:57.088065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 444/897] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:13:56 2022] PUT /identity/v3/projects/73f224d1387f42d988141cc67de12919/users/794de010289243889e61b8c49aa6bc02/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:13:59.166662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc227f0-e5d2-4f49-944e-9915f162e59e None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:59.167107 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc227f0-e5d2-4f49-944e-9915f162e59e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:59.167215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc227f0-e5d2-4f49-944e-9915f162e59e None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:59.168568 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 454/898] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:13:59.174295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579b419-2941-48fe-8612-6fdab282f0f6 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:59.174687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579b419-2941-48fe-8612-6fdab282f0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:59.174826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3579b419-2941-48fe-8612-6fdab282f0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:59.192334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-3579b419-2941-48fe-8612-6fdab282f0f6 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:13:59.193490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3579b419-2941-48fe-8612-6fdab282f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:13:59.258972 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 445/899] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:13:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:13:59.522078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc227f0-e5d2-4f49-944e-9915f162e59e None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:59.619572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc227f0-e5d2-4f49-944e-9915f162e59e None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:59.651718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51aae-4542-4e86-b0af-09d33bf39e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:59.654274 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51aae-4542-4e86-b0af-09d33bf39e04 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:59.654274 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51aae-4542-4e86-b0af-09d33bf39e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:59.654274 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51aae-4542-4e86-b0af-09d33bf39e04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:59.662619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b51aae-4542-4e86-b0af-09d33bf39e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-69b51aae-4542-4e86-b0af-09d33bf39e04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:13:59.667345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:13:59.668633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:13:59.669788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 455/900] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:13:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:13:59.674298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579b419-2941-48fe-8612-6fdab282f0f6 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:59.767821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3579b419-2941-48fe-8612-6fdab282f0f6 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:59.780797 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:59.799083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfba06-3af0-4cf3-8321-a1e2fe7d4846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:59.799671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfba06-3af0-4cf3-8321-a1e2fe7d4846 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:59.799946 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfba06-3af0-4cf3-8321-a1e2fe7d4846 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:59.800212 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfba06-3af0-4cf3-8321-a1e2fe7d4846 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:59.800858 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfba06-3af0-4cf3-8321-a1e2fe7d4846 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:13:59.801154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfba06-3af0-4cf3-8321-a1e2fe7d4846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:59.802574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfba06-3af0-4cf3-8321-a1e2fe7d4846 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:59.810103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 446/901] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:59.816443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51aae-4542-4e86-b0af-09d33bf39e04 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:13:59.850937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:13:59.911414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51aae-4542-4e86-b0af-09d33bf39e04 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:13:59.944646 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62e83-056c-429e-acf0-c3b812569aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:13:59.945252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62e83-056c-429e-acf0-c3b812569aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:13:59.945530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62e83-056c-429e-acf0-c3b812569aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:13:59.945825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d62e83-056c-429e-acf0-c3b812569aa8 None admin] PATH_INFO: `/v3/users/794de010289243889e61b8c49aa6bc02` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:13:59.946959 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62e83-056c-429e-acf0-c3b812569aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=794de010289243889e61b8c49aa6bc02)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:13:59.948386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d62e83-056c-429e-acf0-c3b812569aa8 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:13:59.949916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 456/902] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Oct 19 02:13:59 2022] GET /identity/v3/users/794de010289243889e61b8c49aa6bc02 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:13:59.956154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfba06-3af0-4cf3-8321-a1e2fe7d4846 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:14:00.051126 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfba06-3af0-4cf3-8321-a1e2fe7d4846 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:14:00.082558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae730b37-1302-42c3-ad7f-b0ff5ba3e64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:14:00.083146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae730b37-1302-42c3-ad7f-b0ff5ba3e64e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:14:00.083422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae730b37-1302-42c3-ad7f-b0ff5ba3e64e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:14:00.083690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae730b37-1302-42c3-ad7f-b0ff5ba3e64e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:14:00.129623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:14:00.150924 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae730b37-1302-42c3-ad7f-b0ff5ba3e64e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:14:00.152341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae730b37-1302-42c3-ad7f-b0ff5ba3e64e None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:14:00.245189 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 447/903] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:13:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 02:14:00.251147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62e83-056c-429e-acf0-c3b812569aa8 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:14:00.346436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d62e83-056c-429e-acf0-c3b812569aa8 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:14:00.348602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:14:00.378203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e8d75-b21c-4bb9-a466-656f54f70488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:14:00.378811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e8d75-b21c-4bb9-a466-656f54f70488 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:14:00.379089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e8d75-b21c-4bb9-a466-656f54f70488 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:14:00.379360 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e8d75-b21c-4bb9-a466-656f54f70488 None admin] PATH_INFO: `/v3/projects/73f224d1387f42d988141cc67de12919` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:14:00.380463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e8d75-b21c-4bb9-a466-656f54f70488 None admin] RBAC: Authorizing `identity:get_project(project_id=73f224d1387f42d988141cc67de12919)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:14:00.381888 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e8d75-b21c-4bb9-a466-656f54f70488 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:14:00.383730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 457/904] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Oct 19 02:14:00 2022] GET /identity/v3/projects/73f224d1387f42d988141cc67de12919 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:14:00.389414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae730b37-1302-42c3-ad7f-b0ff5ba3e64e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:14:00.461543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:14:00.483272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae730b37-1302-42c3-ad7f-b0ff5ba3e64e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:14:00.515643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1994bd-a6d3-49ed-823e-9339f62ca699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:14:00.516728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1994bd-a6d3-49ed-823e-9339f62ca699 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:14:00.517000 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1994bd-a6d3-49ed-823e-9339f62ca699 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:14:00.517268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1994bd-a6d3-49ed-823e-9339f62ca699 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:14:00.517981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1994bd-a6d3-49ed-823e-9339f62ca699 None admin] RBAC: Adding query filter params (role.id=d5070bb151d343ca8de32abe373f4a73, scope.project.id=73f224d1387f42d988141cc67de12919, user.id=794de010289243889e61b8c49aa6bc02) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:14:00.518314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1994bd-a6d3-49ed-823e-9339f62ca699 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:14:00.519721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1994bd-a6d3-49ed-823e-9339f62ca699 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:14:00.525207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 448/905] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:14:00 2022] GET /identity/v3/role_assignments?user.id=794de010289243889e61b8c49aa6bc02&scope.project.id=73f224d1387f42d988141cc67de12919&role.id=d5070bb151d343ca8de32abe373f4a73 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:14:10.384715 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=14 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:14:10.522772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=30 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:19.506672 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d026adf-21cb-4a96-bf3d-e2a07428b7c4 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:19.508819 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d026adf-21cb-4a96-bf3d-e2a07428b7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:19.508819 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d026adf-21cb-4a96-bf3d-e2a07428b7c4 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:19.508819 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 458/906] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:15:19.513848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767d056-c5a6-4da2-a50b-06f6f457b27d None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:19.514276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767d056-c5a6-4da2-a50b-06f6f457b27d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:19.514395 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767d056-c5a6-4da2-a50b-06f6f457b27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:19.532508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-3767d056-c5a6-4da2-a50b-06f6f457b27d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:15:19.536918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3767d056-c5a6-4da2-a50b-06f6f457b27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:15:19.596672 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 449/907] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:15:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:15:19.863528 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d026adf-21cb-4a96-bf3d-e2a07428b7c4 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:19.961913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d026adf-21cb-4a96-bf3d-e2a07428b7c4 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:19.995616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35132d-9bd4-45dd-a249-0f5a661c5a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:19.996229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35132d-9bd4-45dd-a249-0f5a661c5a28 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:19.996504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35132d-9bd4-45dd-a249-0f5a661c5a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:19.996770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35132d-9bd4-45dd-a249-0f5a661c5a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:20.006426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35132d-9bd4-45dd-a249-0f5a661c5a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:20.007841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35132d-9bd4-45dd-a249-0f5a661c5a28 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-fe35132d-9bd4-45dd-a249-0f5a661c5a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:15:20.021789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:20.028772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:15:20.029973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:15:20.029973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:20.029973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:15:20.029973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 459/908] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:20.030293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767d056-c5a6-4da2-a50b-06f6f457b27d None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:20.125587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767d056-c5a6-4da2-a50b-06f6f457b27d None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:20.152337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:20.157911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3c6dd-3b0b-4cbc-935e-96c83a8d165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:20.158540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3c6dd-3b0b-4cbc-935e-96c83a8d165f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:20.158815 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3c6dd-3b0b-4cbc-935e-96c83a8d165f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:20.159081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3c6dd-3b0b-4cbc-935e-96c83a8d165f None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:20.159715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3c6dd-3b0b-4cbc-935e-96c83a8d165f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:20.160010 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3c6dd-3b0b-4cbc-935e-96c83a8d165f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:20.161908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3c6dd-3b0b-4cbc-935e-96c83a8d165f None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:20.172672 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 450/909] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:20.179011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35132d-9bd4-45dd-a249-0f5a661c5a28 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:20.230787 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:20.275492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35132d-9bd4-45dd-a249-0f5a661c5a28 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:20.307594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5cac9-d405-4162-ad20-f81e09509c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:20.308183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5cac9-d405-4162-ad20-f81e09509c26 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:20.308458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5cac9-d405-4162-ad20-f81e09509c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:20.308726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5cac9-d405-4162-ad20-f81e09509c26 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:20.309379 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5cac9-d405-4162-ad20-f81e09509c26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:20.310852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5cac9-d405-4162-ad20-f81e09509c26 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:20.410507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 460/910] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Oct 19 02:15:20 2022] POST /identity/v3/users => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:15:22.575649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfa34d-b2bb-455b-8400-590db4850565 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:22.576195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfa34d-b2bb-455b-8400-590db4850565 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:22.576195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfa34d-b2bb-455b-8400-590db4850565 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:22.577517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 451/911] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:15:22.582608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243a7be-0274-42a0-9a22-52ef2fb42cca None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:22.582999 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243a7be-0274-42a0-9a22-52ef2fb42cca None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:22.583676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243a7be-0274-42a0-9a22-52ef2fb42cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:22.602302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-1243a7be-0274-42a0-9a22-52ef2fb42cca None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:15:22.603592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1243a7be-0274-42a0-9a22-52ef2fb42cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:15:22.669043 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 461/912] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:15:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:15:22.941031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfa34d-b2bb-455b-8400-590db4850565 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:23.040690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfa34d-b2bb-455b-8400-590db4850565 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:23.072780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036f5d9-cf85-4106-b924-a5ce11d6a55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:23.073402 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036f5d9-cf85-4106-b924-a5ce11d6a55f None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:23.073683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036f5d9-cf85-4106-b924-a5ce11d6a55f None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:23.073956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036f5d9-cf85-4106-b924-a5ce11d6a55f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:23.084790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036f5d9-cf85-4106-b924-a5ce11d6a55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-9036f5d9-cf85-4106-b924-a5ce11d6a55f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:15:23.089608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:23.095251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:23.096239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 452/913] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:15:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:23.097414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243a7be-0274-42a0-9a22-52ef2fb42cca None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:23.135975 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:23.191578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243a7be-0274-42a0-9a22-52ef2fb42cca None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:23.225468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3765c1a-e9a2-46bb-b607-9df52a6d1bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:23.226091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3765c1a-e9a2-46bb-b607-9df52a6d1bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:23.226372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3765c1a-e9a2-46bb-b607-9df52a6d1bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:23.226641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3765c1a-e9a2-46bb-b607-9df52a6d1bc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:23.227298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3765c1a-e9a2-46bb-b607-9df52a6d1bc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:23.227587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3765c1a-e9a2-46bb-b607-9df52a6d1bc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:23.228935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3765c1a-e9a2-46bb-b607-9df52a6d1bc0 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:23.236590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 462/914] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:15:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:23.242853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036f5d9-cf85-4106-b924-a5ce11d6a55f None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:23.291623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:23.338768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036f5d9-cf85-4106-b924-a5ce11d6a55f None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:23.376742 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cf9d4-b5ed-4c07-8e22-8f503d3c4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:23.377372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cf9d4-b5ed-4c07-8e22-8f503d3c4174 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:23.377643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cf9d4-b5ed-4c07-8e22-8f503d3c4174 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:23.377907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cf9d4-b5ed-4c07-8e22-8f503d3c4174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:23.388857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cf9d4-b5ed-4c07-8e22-8f503d3c4174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:23.393178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cf9d4-b5ed-4c07-8e22-8f503d3c4174 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-1f5cf9d4-b5ed-4c07-8e22-8f503d3c4174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:15:23.404219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:23.409583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:15:23.410969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:15:23.410969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:23.410969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:23.410969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 453/915] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:23.411270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3765c1a-e9a2-46bb-b607-9df52a6d1bc0 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:23.505843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:23.508696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3765c1a-e9a2-46bb-b607-9df52a6d1bc0 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:23.541475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711d027-2580-41d1-83ea-914a113dc63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:23.542113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711d027-2580-41d1-83ea-914a113dc63e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:23.542394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711d027-2580-41d1-83ea-914a113dc63e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:23.542664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711d027-2580-41d1-83ea-914a113dc63e None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:23.543301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711d027-2580-41d1-83ea-914a113dc63e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:23.543599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711d027-2580-41d1-83ea-914a113dc63e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:23.544958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711d027-2580-41d1-83ea-914a113dc63e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:23.556594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 463/916] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:23.562359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cf9d4-b5ed-4c07-8e22-8f503d3c4174 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:23.663540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cf9d4-b5ed-4c07-8e22-8f503d3c4174 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:23.665016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:23.696091 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce56d34-e871-4212-9fec-e65d8186b116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:23.696701 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce56d34-e871-4212-9fec-e65d8186b116 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:23.696971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce56d34-e871-4212-9fec-e65d8186b116 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:23.697232 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce56d34-e871-4212-9fec-e65d8186b116 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:23.705316 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce56d34-e871-4212-9fec-e65d8186b116 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:23.706779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce56d34-e871-4212-9fec-e65d8186b116 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-dce56d34-e871-4212-9fec-e65d8186b116 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:15:23.717949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:23.722750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:23.724621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:23.724621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:23.724621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:15:23.724621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:15:23.724621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:15:23.724621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:15:23.724621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:15:23.724621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:15:23.724621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 19 02:15:23.724621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:23.724621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 454/917] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:23.726195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711d027-2580-41d1-83ea-914a113dc63e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:23.823117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711d027-2580-41d1-83ea-914a113dc63e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:23.856205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424920c-5e7a-4d02-8763-779ea307eafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:23.856798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424920c-5e7a-4d02-8763-779ea307eafc None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:23.857077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424920c-5e7a-4d02-8763-779ea307eafc None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:23.857351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424920c-5e7a-4d02-8763-779ea307eafc None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:23.858089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424920c-5e7a-4d02-8763-779ea307eafc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:23.858397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424920c-5e7a-4d02-8763-779ea307eafc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:23.859812 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424920c-5e7a-4d02-8763-779ea307eafc None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:23.877111 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 464/918] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Oct 19 02:15:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:23.882894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce56d34-e871-4212-9fec-e65d8186b116 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:23.978705 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce56d34-e871-4212-9fec-e65d8186b116 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:24.010215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37047135-069b-45a7-a11b-344c1f3f061c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:24.010820 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37047135-069b-45a7-a11b-344c1f3f061c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:24.011092 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37047135-069b-45a7-a11b-344c1f3f061c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:24.011357 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37047135-069b-45a7-a11b-344c1f3f061c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:24.021917 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37047135-069b-45a7-a11b-344c1f3f061c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:24.023414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37047135-069b-45a7-a11b-344c1f3f061c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-37047135-069b-45a7-a11b-344c1f3f061c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:15:24.037841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:24.039049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:15:24.040209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:15:24.040209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:24.040209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:24.040209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 455/919] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:24.044843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424920c-5e7a-4d02-8763-779ea307eafc None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:24.063309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:24.141917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424920c-5e7a-4d02-8763-779ea307eafc None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:24.176405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47381eda-a286-403d-9150-2edc10d17ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:24.177041 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47381eda-a286-403d-9150-2edc10d17ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:24.177281 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47381eda-a286-403d-9150-2edc10d17ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:24.177551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47381eda-a286-403d-9150-2edc10d17ae5 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:24.178218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47381eda-a286-403d-9150-2edc10d17ae5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:24.178549 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47381eda-a286-403d-9150-2edc10d17ae5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:24.179969 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47381eda-a286-403d-9150-2edc10d17ae5 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:24.190800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 465/920] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:24.196583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37047135-069b-45a7-a11b-344c1f3f061c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:24.213424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:24.293461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37047135-069b-45a7-a11b-344c1f3f061c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:24.324939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b562a68-cbdf-48d7-9c92-9b3c65e13a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:24.335429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b562a68-cbdf-48d7-9c92-9b3c65e13a20 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:24.335429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b562a68-cbdf-48d7-9c92-9b3c65e13a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:24.335429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b562a68-cbdf-48d7-9c92-9b3c65e13a20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:24.335429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b562a68-cbdf-48d7-9c92-9b3c65e13a20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:24.335429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b562a68-cbdf-48d7-9c92-9b3c65e13a20 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-5b562a68-cbdf-48d7-9c92-9b3c65e13a20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:15:24.345214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:24.346173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:24.347060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:15:24.347060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:15:24.347060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:15:24.347060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:15:24.347060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:15:24.347060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:15:24.347060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:15:24.347060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:24.347060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 456/921] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:15:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:24.353109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47381eda-a286-403d-9150-2edc10d17ae5 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:24.429172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:24.451065 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47381eda-a286-403d-9150-2edc10d17ae5 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:24.483583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ba303-1e0a-4cd8-b675-0fd54920afe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:24.484170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ba303-1e0a-4cd8-b675-0fd54920afe6 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:24.484445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ba303-1e0a-4cd8-b675-0fd54920afe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:24.484714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ba303-1e0a-4cd8-b675-0fd54920afe6 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:24.485384 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ba303-1e0a-4cd8-b675-0fd54920afe6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:24.485681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ba303-1e0a-4cd8-b675-0fd54920afe6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:24.487113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ba303-1e0a-4cd8-b675-0fd54920afe6 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:24.498569 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 466/922] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:24.504646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b562a68-cbdf-48d7-9c92-9b3c65e13a20 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:24.577766 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:24.601510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b562a68-cbdf-48d7-9c92-9b3c65e13a20 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:24.633454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b0fa6-a4ae-4c97-8b20-387bb2fc6e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:24.634124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b0fa6-a4ae-4c97-8b20-387bb2fc6e05 None admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:24.634401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b0fa6-a4ae-4c97-8b20-387bb2fc6e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:24.634666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b0fa6-a4ae-4c97-8b20-387bb2fc6e05 None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/9243ebe24ae84157b74696c29bda3279/roles/695bad5e383d4052a3d6ffa680329cb7` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:24.649711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b0fa6-a4ae-4c97-8b20-387bb2fc6e05 None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=9243ebe24ae84157b74696c29bda3279, role_id=695bad5e383d4052a3d6ffa680329cb7)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:24.651171 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b0fa6-a4ae-4c97-8b20-387bb2fc6e05 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:24.719745 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 457/923] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:15:24 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/9243ebe24ae84157b74696c29bda3279/roles/695bad5e383d4052a3d6ffa680329cb7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:15:26.940197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b589072-ea45-43d4-b6ad-2cd53b8391c2 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:26.940731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b589072-ea45-43d4-b6ad-2cd53b8391c2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:26.940731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b589072-ea45-43d4-b6ad-2cd53b8391c2 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:26.941949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 467/924] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:15:26.947714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2bef6-a441-4fde-aef9-6ef730b507c9 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:26.948150 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2bef6-a441-4fde-aef9-6ef730b507c9 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:26.948379 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2bef6-a441-4fde-aef9-6ef730b507c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:26.966628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-e6b2bef6-a441-4fde-aef9-6ef730b507c9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:15:26.967799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e6b2bef6-a441-4fde-aef9-6ef730b507c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:15:27.032266 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 458/925] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:15:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:15:27.299589 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b589072-ea45-43d4-b6ad-2cd53b8391c2 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:27.406163 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b589072-ea45-43d4-b6ad-2cd53b8391c2 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:27.438986 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c62954-9d07-47fe-b72d-984705e9ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:27.439598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c62954-9d07-47fe-b72d-984705e9ac4a None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:27.439874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c62954-9d07-47fe-b72d-984705e9ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:27.440144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c62954-9d07-47fe-b72d-984705e9ac4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:27.446735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:27.450457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c62954-9d07-47fe-b72d-984705e9ac4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-34c62954-9d07-47fe-b72d-984705e9ac4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:15:27.455137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:27.456251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:15:27.457067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 468/926] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Oct 19 02:15:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:27.462323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2bef6-a441-4fde-aef9-6ef730b507c9 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:27.515043 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:27.557262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2bef6-a441-4fde-aef9-6ef730b507c9 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:27.589930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fb7f4-516b-4473-a744-33973a2b91d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:27.590568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fb7f4-516b-4473-a744-33973a2b91d0 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:27.590847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fb7f4-516b-4473-a744-33973a2b91d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:27.591126 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fb7f4-516b-4473-a744-33973a2b91d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:27.591789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fb7f4-516b-4473-a744-33973a2b91d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:27.592091 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fb7f4-516b-4473-a744-33973a2b91d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:27.594283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fb7f4-516b-4473-a744-33973a2b91d0 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:27.603113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 459/927] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Oct 19 02:15:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:27.609142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c62954-9d07-47fe-b72d-984705e9ac4a None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:27.705793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c62954-9d07-47fe-b72d-984705e9ac4a None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:27.739561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d991ab-a6ed-4e39-94b1-e50ed0d17d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:27.740174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d991ab-a6ed-4e39-94b1-e50ed0d17d84 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:27.740450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d991ab-a6ed-4e39-94b1-e50ed0d17d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:27.740716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d991ab-a6ed-4e39-94b1-e50ed0d17d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:27.750447 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d991ab-a6ed-4e39-94b1-e50ed0d17d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:27.751870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d991ab-a6ed-4e39-94b1-e50ed0d17d84 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-44d991ab-a6ed-4e39-94b1-e50ed0d17d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:15:27.766461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:27.767615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:15:27.768729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:15:27.768729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:27.768729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:15:27.770200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 469/928] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:27.773951 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fb7f4-516b-4473-a744-33973a2b91d0 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:27.871590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fb7f4-516b-4473-a744-33973a2b91d0 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:27.887195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:27.904371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e46e9a-0ca0-4ae0-b772-5284dce888c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:27.904972 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e46e9a-0ca0-4ae0-b772-5284dce888c8 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:27.905250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e46e9a-0ca0-4ae0-b772-5284dce888c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:27.905521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e46e9a-0ca0-4ae0-b772-5284dce888c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:27.906193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e46e9a-0ca0-4ae0-b772-5284dce888c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:27.906497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e46e9a-0ca0-4ae0-b772-5284dce888c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:27.907857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e46e9a-0ca0-4ae0-b772-5284dce888c8 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:27.919771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 460/929] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:27.925588 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d991ab-a6ed-4e39-94b1-e50ed0d17d84 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:27.973662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:28.022632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d991ab-a6ed-4e39-94b1-e50ed0d17d84 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:28.055439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b12252-0cb8-4273-9240-2e0f705df53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:28.056050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b12252-0cb8-4273-9240-2e0f705df53e None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:28.056326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b12252-0cb8-4273-9240-2e0f705df53e None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:28.056596 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b12252-0cb8-4273-9240-2e0f705df53e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:28.063788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b12252-0cb8-4273-9240-2e0f705df53e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:28.065210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b12252-0cb8-4273-9240-2e0f705df53e None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-11b12252-0cb8-4273-9240-2e0f705df53e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:15:28.076301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:28.078856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:28.079813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:28.079813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:28.079813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:15:28.079813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:15:28.079813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:15:28.079813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:15:28.079813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:15:28.079813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:15:28.079813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 19 02:15:28.079813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:15:28.079813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 470/930] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:28.084039 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e46e9a-0ca0-4ae0-b772-5284dce888c8 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:28.180622 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e46e9a-0ca0-4ae0-b772-5284dce888c8 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:28.213963 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200f6dc-93a6-4cec-b667-beb9d172d3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:28.214586 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200f6dc-93a6-4cec-b667-beb9d172d3ee None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:28.214864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200f6dc-93a6-4cec-b667-beb9d172d3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:28.215137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200f6dc-93a6-4cec-b667-beb9d172d3ee None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:28.215855 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200f6dc-93a6-4cec-b667-beb9d172d3ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:28.216156 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200f6dc-93a6-4cec-b667-beb9d172d3ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:28.217563 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200f6dc-93a6-4cec-b667-beb9d172d3ee None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:28.234999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 461/931] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Oct 19 02:15:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:28.240706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b12252-0cb8-4273-9240-2e0f705df53e None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:28.337556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b12252-0cb8-4273-9240-2e0f705df53e None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:28.347831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:28.369879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6df860-6e7f-4bfd-a20b-b5e978c915f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:28.370501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6df860-6e7f-4bfd-a20b-b5e978c915f3 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:28.370780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6df860-6e7f-4bfd-a20b-b5e978c915f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:28.371050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6df860-6e7f-4bfd-a20b-b5e978c915f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:28.434184 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6df860-6e7f-4bfd-a20b-b5e978c915f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:28.435558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6df860-6e7f-4bfd-a20b-b5e978c915f3 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:28.530228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 471/932] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:15:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 02:15:28.536058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200f6dc-93a6-4cec-b667-beb9d172d3ee None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:28.569642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:28.632663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200f6dc-93a6-4cec-b667-beb9d172d3ee None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:28.664305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a5f1f-ed7b-487d-9236-057b2b3fac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:28.664920 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a5f1f-ed7b-487d-9236-057b2b3fac0e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:28.665195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a5f1f-ed7b-487d-9236-057b2b3fac0e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:28.665463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a5f1f-ed7b-487d-9236-057b2b3fac0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:28.674946 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a5f1f-ed7b-487d-9236-057b2b3fac0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:28.676361 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a5f1f-ed7b-487d-9236-057b2b3fac0e None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-a14a5f1f-ed7b-487d-9236-057b2b3fac0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:15:28.690680 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:28.691761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:15:28.693421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:15:28.693421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:28.693421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:28.693421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 462/933] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:28.697882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6df860-6e7f-4bfd-a20b-b5e978c915f3 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:28.709118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:28.791703 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6df860-6e7f-4bfd-a20b-b5e978c915f3 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:28.826618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9c1d9-a27b-42fd-a194-0ff1ddd0ec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:28.827212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9c1d9-a27b-42fd-a194-0ff1ddd0ec38 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:28.827487 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9c1d9-a27b-42fd-a194-0ff1ddd0ec38 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:28.827760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9c1d9-a27b-42fd-a194-0ff1ddd0ec38 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:28.828388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9c1d9-a27b-42fd-a194-0ff1ddd0ec38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:28.828686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9c1d9-a27b-42fd-a194-0ff1ddd0ec38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:28.830036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9c1d9-a27b-42fd-a194-0ff1ddd0ec38 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:28.840838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 472/934] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:28.846431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a5f1f-ed7b-487d-9236-057b2b3fac0e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:28.928893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:28.942615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a5f1f-ed7b-487d-9236-057b2b3fac0e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:28.974554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c83b3-120e-407d-9529-145b2685fd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:28.975169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c83b3-120e-407d-9529-145b2685fd0e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:28.975446 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c83b3-120e-407d-9529-145b2685fd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:28.975716 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c83b3-120e-407d-9529-145b2685fd0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:28.982148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c83b3-120e-407d-9529-145b2685fd0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:28.983568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c83b3-120e-407d-9529-145b2685fd0e None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-422c83b3-120e-407d-9529-145b2685fd0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:15:28.995061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:28.996121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:28.997501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:15:28.997501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:15:28.997501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:15:28.997501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:15:28.997501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:15:28.997501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:15:28.997501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:15:28.997501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:28.997501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 463/935] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:29.001958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9c1d9-a27b-42fd-a194-0ff1ddd0ec38 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:29.067838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:29.096384 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9c1d9-a27b-42fd-a194-0ff1ddd0ec38 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:29.130028 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f60e5-3405-4a03-ba78-3879b7e05b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:29.130652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f60e5-3405-4a03-ba78-3879b7e05b52 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:29.130934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f60e5-3405-4a03-ba78-3879b7e05b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:29.131195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f60e5-3405-4a03-ba78-3879b7e05b52 None admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:29.131871 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f60e5-3405-4a03-ba78-3879b7e05b52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:29.132172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f60e5-3405-4a03-ba78-3879b7e05b52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:29.133639 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f60e5-3405-4a03-ba78-3879b7e05b52 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:29.145053 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 473/936] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:15:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:29.151143 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c83b3-120e-407d-9529-145b2685fd0e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:29.248798 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c83b3-120e-407d-9529-145b2685fd0e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:29.280878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933fe51-6b19-4e0e-9da8-00b982a51710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:29.281471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933fe51-6b19-4e0e-9da8-00b982a51710 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:29.281749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933fe51-6b19-4e0e-9da8-00b982a51710 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:29.282018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933fe51-6b19-4e0e-9da8-00b982a51710 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:29.282770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933fe51-6b19-4e0e-9da8-00b982a51710 None admin] RBAC: Adding query filter params (role.id=695bad5e383d4052a3d6ffa680329cb7, scope.project.id=676e986ff972410d9f498800d313476f, user.id=9243ebe24ae84157b74696c29bda3279) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:29.283079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933fe51-6b19-4e0e-9da8-00b982a51710 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:29.284489 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933fe51-6b19-4e0e-9da8-00b982a51710 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:29.290661 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 464/937] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:15:29 2022] GET /identity/v3/role_assignments?user.id=9243ebe24ae84157b74696c29bda3279&scope.project.id=676e986ff972410d9f498800d313476f&role.id=695bad5e383d4052a3d6ffa680329cb7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:31.439515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45950232-ea0c-4740-8151-401ab9ea7c64 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:31.440051 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45950232-ea0c-4740-8151-401ab9ea7c64 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:31.440051 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45950232-ea0c-4740-8151-401ab9ea7c64 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:31.441296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 474/938] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:15:31.446502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfae2e8-7e0b-4294-90a0-1ea960b991c0 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:31.446893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfae2e8-7e0b-4294-90a0-1ea960b991c0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:31.447085 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfae2e8-7e0b-4294-90a0-1ea960b991c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:31.465305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7dfae2e8-7e0b-4294-90a0-1ea960b991c0 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:15:31.466523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7dfae2e8-7e0b-4294-90a0-1ea960b991c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:15:31.469183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:31.530697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 465/939] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:15:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:15:31.796793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45950232-ea0c-4740-8151-401ab9ea7c64 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:31.897253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45950232-ea0c-4740-8151-401ab9ea7c64 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:31.924203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:31.929822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53cc1e-790d-4cc7-9384-bae411de1186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:31.932719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53cc1e-790d-4cc7-9384-bae411de1186 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:31.932719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53cc1e-790d-4cc7-9384-bae411de1186 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:31.932719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53cc1e-790d-4cc7-9384-bae411de1186 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:31.940834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53cc1e-790d-4cc7-9384-bae411de1186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-6f53cc1e-790d-4cc7-9384-bae411de1186 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:15:31.946215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:31.947359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:15:31.948260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 475/940] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:31.953468 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfae2e8-7e0b-4294-90a0-1ea960b991c0 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:32.048775 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfae2e8-7e0b-4294-90a0-1ea960b991c0 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:32.081748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138dcfb-7530-454b-8789-360538a6ff73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:32.082369 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138dcfb-7530-454b-8789-360538a6ff73 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:32.082662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138dcfb-7530-454b-8789-360538a6ff73 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:32.082933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138dcfb-7530-454b-8789-360538a6ff73 None admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:32.084181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138dcfb-7530-454b-8789-360538a6ff73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:32.084483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138dcfb-7530-454b-8789-360538a6ff73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:32.085839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138dcfb-7530-454b-8789-360538a6ff73 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:32.093669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 466/941] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:15:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:32.099870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53cc1e-790d-4cc7-9384-bae411de1186 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:32.196656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53cc1e-790d-4cc7-9384-bae411de1186 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:32.230268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5033b0f-b5b2-4f9f-8c10-c475c45c3d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:32.230877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5033b0f-b5b2-4f9f-8c10-c475c45c3d68 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:32.231148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5033b0f-b5b2-4f9f-8c10-c475c45c3d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:32.231415 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5033b0f-b5b2-4f9f-8c10-c475c45c3d68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:32.241470 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5033b0f-b5b2-4f9f-8c10-c475c45c3d68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:32.242991 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5033b0f-b5b2-4f9f-8c10-c475c45c3d68 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-f5033b0f-b5b2-4f9f-8c10-c475c45c3d68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:15:32.257493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:32.258505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:15:32.259409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:15:32.259409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:32.259409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:15:32.259409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 476/942] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:32.264313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138dcfb-7530-454b-8789-360538a6ff73 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:32.267296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:32.359513 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138dcfb-7530-454b-8789-360538a6ff73 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:32.393036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f04314-9479-4bdd-ae1c-038db7a66512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:32.393630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f04314-9479-4bdd-ae1c-038db7a66512 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:32.393905 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f04314-9479-4bdd-ae1c-038db7a66512 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:32.394204 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f04314-9479-4bdd-ae1c-038db7a66512 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:32.394837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f04314-9479-4bdd-ae1c-038db7a66512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:32.395133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f04314-9479-4bdd-ae1c-038db7a66512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:32.396478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f04314-9479-4bdd-ae1c-038db7a66512 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:32.408126 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 467/943] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:32.413960 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5033b0f-b5b2-4f9f-8c10-c475c45c3d68 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:32.456963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:32.509988 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5033b0f-b5b2-4f9f-8c10-c475c45c3d68 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:32.543852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443c47c-e31e-4a2e-ac55-d2261fa94c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:32.544459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443c47c-e31e-4a2e-ac55-d2261fa94c42 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:32.544733 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443c47c-e31e-4a2e-ac55-d2261fa94c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:32.544999 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443c47c-e31e-4a2e-ac55-d2261fa94c42 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:32.551798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443c47c-e31e-4a2e-ac55-d2261fa94c42 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:32.553747 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443c47c-e31e-4a2e-ac55-d2261fa94c42 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-e443c47c-e31e-4a2e-ac55-d2261fa94c42 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:15:32.564702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:32.565887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:32.566887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:32.566887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:32.566887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:15:32.566887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:15:32.566887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:15:32.566887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:15:32.566887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:15:32.566887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:15:32.566887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 19 02:15:32.566887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:15:32.566887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 477/944] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:32.572124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f04314-9479-4bdd-ae1c-038db7a66512 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:32.630917 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:32.666848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f04314-9479-4bdd-ae1c-038db7a66512 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:32.698754 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd954fe-e9b0-4110-a724-264dc2167e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:32.699368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd954fe-e9b0-4110-a724-264dc2167e0d None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:32.699642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd954fe-e9b0-4110-a724-264dc2167e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:32.699908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd954fe-e9b0-4110-a724-264dc2167e0d None admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:32.700611 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd954fe-e9b0-4110-a724-264dc2167e0d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:32.700950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd954fe-e9b0-4110-a724-264dc2167e0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:32.702374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd954fe-e9b0-4110-a724-264dc2167e0d None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:32.718377 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 468/945] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Oct 19 02:15:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:32.724966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443c47c-e31e-4a2e-ac55-d2261fa94c42 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:32.821014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443c47c-e31e-4a2e-ac55-d2261fa94c42 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:32.822515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:32.854051 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53265d-a566-4689-859e-e48deca9851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:32.854688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53265d-a566-4689-859e-e48deca9851f None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:32.854967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53265d-a566-4689-859e-e48deca9851f None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:32.855242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53265d-a566-4689-859e-e48deca9851f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:32.865326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53265d-a566-4689-859e-e48deca9851f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:32.866790 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53265d-a566-4689-859e-e48deca9851f None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-ec53265d-a566-4689-859e-e48deca9851f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:15:32.880656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:32.881857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:15:32.883044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:15:32.883044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:32.883044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:15:32.883044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 478/946] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:32.887452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd954fe-e9b0-4110-a724-264dc2167e0d None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:32.983501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd954fe-e9b0-4110-a724-264dc2167e0d None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:33.002996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:33.016112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bf979-2a47-423e-9450-7610a4f71c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:33.016706 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bf979-2a47-423e-9450-7610a4f71c8b None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:33.016980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bf979-2a47-423e-9450-7610a4f71c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:33.017247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52bf979-2a47-423e-9450-7610a4f71c8b None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:33.017877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bf979-2a47-423e-9450-7610a4f71c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:33.018212 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bf979-2a47-423e-9450-7610a4f71c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:33.019635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52bf979-2a47-423e-9450-7610a4f71c8b None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:33.030791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 469/947] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:33.036567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53265d-a566-4689-859e-e48deca9851f None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:33.135416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53265d-a566-4689-859e-e48deca9851f None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:33.168359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff90038-a803-418c-bf17-1e080eaecead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:33.169093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff90038-a803-418c-bf17-1e080eaecead None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:33.169369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff90038-a803-418c-bf17-1e080eaecead None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:33.169636 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff90038-a803-418c-bf17-1e080eaecead None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:33.176071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff90038-a803-418c-bf17-1e080eaecead None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:33.177503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff90038-a803-418c-bf17-1e080eaecead None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-cff90038-a803-418c-bf17-1e080eaecead None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:15:33.188445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:33.189521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:33.190674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:15:33.190674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:15:33.190674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:15:33.190674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:15:33.190674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:15:33.190674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:15:33.190674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:15:33.190674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:15:33.190674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 479/948] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:15:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:33.195430 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bf979-2a47-423e-9450-7610a4f71c8b None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:33.293486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52bf979-2a47-423e-9450-7610a4f71c8b None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:33.325523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4d232-73e3-4090-ae6b-f91ee26badb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:33.326175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4d232-73e3-4090-ae6b-f91ee26badb3 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:33.326452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4d232-73e3-4090-ae6b-f91ee26badb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:33.326717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4d232-73e3-4090-ae6b-f91ee26badb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:33.327389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4d232-73e3-4090-ae6b-f91ee26badb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:33.327687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4d232-73e3-4090-ae6b-f91ee26badb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:33.329115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4d232-73e3-4090-ae6b-f91ee26badb3 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:33.341196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 470/949] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:15:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:33.347005 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff90038-a803-418c-bf17-1e080eaecead None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:33.363484 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:33.446878 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff90038-a803-418c-bf17-1e080eaecead None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:33.479057 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8f61d-10aa-4517-be2b-5792b9427763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:33.479695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8f61d-10aa-4517-be2b-5792b9427763 None admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:33.483637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8f61d-10aa-4517-be2b-5792b9427763 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:33.483637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8f61d-10aa-4517-be2b-5792b9427763 None admin] PATH_INFO: `/v3/projects/676e986ff972410d9f498800d313476f/users/9243ebe24ae84157b74696c29bda3279/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:33.483637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8f61d-10aa-4517-be2b-5792b9427763 None admin] RBAC: Authorizing `identity:create_grant(project_id=676e986ff972410d9f498800d313476f, user_id=9243ebe24ae84157b74696c29bda3279, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:33.483637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8f61d-10aa-4517-be2b-5792b9427763 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:33.582142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 480/950] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Oct 19 02:15:33 2022] PUT /identity/v3/projects/676e986ff972410d9f498800d313476f/users/9243ebe24ae84157b74696c29bda3279/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:15:35.703853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9415c2b-86ce-4525-b5a3-202f553bf404 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:35.704415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9415c2b-86ce-4525-b5a3-202f553bf404 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:35.704415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9415c2b-86ce-4525-b5a3-202f553bf404 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:35.705699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 471/951] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:15:35.711165 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0df85-b707-4b75-86d1-2125ded68ad3 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:35.711581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0df85-b707-4b75-86d1-2125ded68ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:35.711721 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0df85-b707-4b75-86d1-2125ded68ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:35.730986 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-5ab0df85-b707-4b75-86d1-2125ded68ad3 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:15:35.732167 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5ab0df85-b707-4b75-86d1-2125ded68ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:15:35.796142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 481/952] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:15:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:15:36.063866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9415c2b-86ce-4525-b5a3-202f553bf404 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:36.107133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:36.162608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9415c2b-86ce-4525-b5a3-202f553bf404 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:36.194699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100c95a-addf-429f-8620-0b194c043365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:36.195313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100c95a-addf-429f-8620-0b194c043365 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:36.198143 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100c95a-addf-429f-8620-0b194c043365 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:36.198143 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100c95a-addf-429f-8620-0b194c043365 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:36.205487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100c95a-addf-429f-8620-0b194c043365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-a100c95a-addf-429f-8620-0b194c043365 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 19 02:15:36.210364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:36.211498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:36.212559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 472/953] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Oct 19 02:15:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:36.217454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0df85-b707-4b75-86d1-2125ded68ad3 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:36.220479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:36.311957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0df85-b707-4b75-86d1-2125ded68ad3 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:36.344434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc81db-08d3-494e-8ff5-1957a163e5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:36.344991 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc81db-08d3-494e-8ff5-1957a163e5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:36.345274 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc81db-08d3-494e-8ff5-1957a163e5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:36.345542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc81db-08d3-494e-8ff5-1957a163e5f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:36.346228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc81db-08d3-494e-8ff5-1957a163e5f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:36.346527 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc81db-08d3-494e-8ff5-1957a163e5f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:36.347875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc81db-08d3-494e-8ff5-1957a163e5f1 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:36.356252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 482/954] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Oct 19 02:15:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:36.362339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100c95a-addf-429f-8620-0b194c043365 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:36.455889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:36.458858 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100c95a-addf-429f-8620-0b194c043365 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:36.492768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ac045-53fe-4a79-8f58-cbe1088dffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:36.493454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac045-53fe-4a79-8f58-cbe1088dffb5 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:36.493731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac045-53fe-4a79-8f58-cbe1088dffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:36.494003 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac045-53fe-4a79-8f58-cbe1088dffb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:36.504360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ac045-53fe-4a79-8f58-cbe1088dffb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:36.505864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ac045-53fe-4a79-8f58-cbe1088dffb5 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-883ac045-53fe-4a79-8f58-cbe1088dffb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:15:36.519477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:36.520646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:15:36.521759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:15:36.521759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:36.521759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:36.521759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 473/955] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:36.526627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc81db-08d3-494e-8ff5-1957a163e5f1 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:36.592535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:36.625888 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc81db-08d3-494e-8ff5-1957a163e5f1 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:36.657056 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b2f00-7c40-43d0-a667-8147ea1af7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:36.657644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b2f00-7c40-43d0-a667-8147ea1af7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:36.657916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b2f00-7c40-43d0-a667-8147ea1af7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:36.658214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b2f00-7c40-43d0-a667-8147ea1af7c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:36.658841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b2f00-7c40-43d0-a667-8147ea1af7c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:36.659137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b2f00-7c40-43d0-a667-8147ea1af7c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:36.660540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b2f00-7c40-43d0-a667-8147ea1af7c5 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:36.672668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 483/956] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:36.678950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ac045-53fe-4a79-8f58-cbe1088dffb5 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:36.775678 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ac045-53fe-4a79-8f58-cbe1088dffb5 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:36.808029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d4672-bf18-41bc-b24f-eafaf1c22f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:36.808684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d4672-bf18-41bc-b24f-eafaf1c22f8e None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:36.808961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d4672-bf18-41bc-b24f-eafaf1c22f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:36.809237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d4672-bf18-41bc-b24f-eafaf1c22f8e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:36.816436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d4672-bf18-41bc-b24f-eafaf1c22f8e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:36.817858 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d4672-bf18-41bc-b24f-eafaf1c22f8e None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-336d4672-bf18-41bc-b24f-eafaf1c22f8e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 19 02:15:36.829018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:36.830142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:36.831102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:36.831102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:36.831102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 19 02:15:36.831102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 19 02:15:36.831102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 19 02:15:36.831102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 19 02:15:36.831102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 19 02:15:36.831102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 19 02:15:36.831102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 19 02:15:36.831102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:36.831102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 474/957] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:15:36.836410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b2f00-7c40-43d0-a667-8147ea1af7c5 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:36.931235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b2f00-7c40-43d0-a667-8147ea1af7c5 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:36.955061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:36.963852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aff519-ff62-48fd-b1fc-624463b1b0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:36.964443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aff519-ff62-48fd-b1fc-624463b1b0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:36.964718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aff519-ff62-48fd-b1fc-624463b1b0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:36.964983 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aff519-ff62-48fd-b1fc-624463b1b0c6 None admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:36.965687 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aff519-ff62-48fd-b1fc-624463b1b0c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:36.965989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aff519-ff62-48fd-b1fc-624463b1b0c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:36.967433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aff519-ff62-48fd-b1fc-624463b1b0c6 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:36.983801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 484/958] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Oct 19 02:15:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:36.990430 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d4672-bf18-41bc-b24f-eafaf1c22f8e None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:37.005224 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:37.085249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d4672-bf18-41bc-b24f-eafaf1c22f8e None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:37.120510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6cd35-2797-4e54-b1a2-185643bae4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:37.121150 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6cd35-2797-4e54-b1a2-185643bae4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:37.121431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6cd35-2797-4e54-b1a2-185643bae4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:37.121701 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6cd35-2797-4e54-b1a2-185643bae4d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:37.186859 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6cd35-2797-4e54-b1a2-185643bae4d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:37.188304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6cd35-2797-4e54-b1a2-185643bae4d4 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:37.201195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:37.283702 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 475/959] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Oct 19 02:15:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 19 02:15:37.289618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aff519-ff62-48fd-b1fc-624463b1b0c6 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:37.386723 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aff519-ff62-48fd-b1fc-624463b1b0c6 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:37.422621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cace0a4-ddec-4fdd-b715-bd2714ea9596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:37.423237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cace0a4-ddec-4fdd-b715-bd2714ea9596 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:37.423510 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cace0a4-ddec-4fdd-b715-bd2714ea9596 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:37.423778 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cace0a4-ddec-4fdd-b715-bd2714ea9596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:37.434129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cace0a4-ddec-4fdd-b715-bd2714ea9596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:37.435559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cace0a4-ddec-4fdd-b715-bd2714ea9596 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-0cace0a4-ddec-4fdd-b715-bd2714ea9596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 19 02:15:37.456148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:37.457302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 19 02:15:37.458450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 19 02:15:37.458450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 19 02:15:37.458450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:15:37.458450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 485/960] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:37.459299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6cd35-2797-4e54-b1a2-185643bae4d4 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:37.552055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6cd35-2797-4e54-b1a2-185643bae4d4 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:37.564957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:37.585273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfbd0c-1035-4d01-8851-5fbb706e6fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:37.585870 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfbd0c-1035-4d01-8851-5fbb706e6fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:37.586169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfbd0c-1035-4d01-8851-5fbb706e6fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:37.586437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dfbd0c-1035-4d01-8851-5fbb706e6fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:37.587063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfbd0c-1035-4d01-8851-5fbb706e6fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:37.587359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfbd0c-1035-4d01-8851-5fbb706e6fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:37.588699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dfbd0c-1035-4d01-8851-5fbb706e6fd1 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:37.599585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 476/961] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Oct 19 02:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:37.605631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cace0a4-ddec-4fdd-b715-bd2714ea9596 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:37.640634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:37.701310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cace0a4-ddec-4fdd-b715-bd2714ea9596 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:37.733584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046702c6-c344-43e2-babc-6288cc88f1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:37.734160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046702c6-c344-43e2-babc-6288cc88f1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:37.734465 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046702c6-c344-43e2-babc-6288cc88f1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:37.734731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046702c6-c344-43e2-babc-6288cc88f1f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:37.741615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046702c6-c344-43e2-babc-6288cc88f1f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:37.743080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046702c6-c344-43e2-babc-6288cc88f1f2 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application [None req-046702c6-c344-43e2-babc-6288cc88f1f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 19 02:15:37.755072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:37.762412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:37.763540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 19 02:15:37.763540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 19 02:15:37.763540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 19 02:15:37.763540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 19 02:15:37.763540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 19 02:15:37.763540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 19 02:15:37.763540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 19 02:15:37.763540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: ERROR keystone.server.flask.application Oct 19 02:15:37.763540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 486/962] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Oct 19 02:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 19 02:15:37.764151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfbd0c-1035-4d01-8851-5fbb706e6fd1 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:37.859158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dfbd0c-1035-4d01-8851-5fbb706e6fd1 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:37.891017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b00dc-6775-4876-9a21-2d44b64c3a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:37.891604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b00dc-6775-4876-9a21-2d44b64c3a07 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:37.891877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b00dc-6775-4876-9a21-2d44b64c3a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:37.892142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b00dc-6775-4876-9a21-2d44b64c3a07 None admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:37.892815 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b00dc-6775-4876-9a21-2d44b64c3a07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:37.893111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b00dc-6775-4876-9a21-2d44b64c3a07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:37.894552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b00dc-6775-4876-9a21-2d44b64c3a07 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:37.906483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 477/963] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Oct 19 02:15:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:37.912817 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046702c6-c344-43e2-babc-6288cc88f1f2 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:38.004105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:38.010151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046702c6-c344-43e2-babc-6288cc88f1f2 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:38.042216 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d0dbb-efe0-45b3-b0cc-021f67501059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:38.042804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d0dbb-efe0-45b3-b0cc-021f67501059 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:38.043082 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d0dbb-efe0-45b3-b0cc-021f67501059 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:38.043352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d0dbb-efe0-45b3-b0cc-021f67501059 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:38.044069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d0dbb-efe0-45b3-b0cc-021f67501059 None admin] RBAC: Adding query filter params (role.id=d5070bb151d343ca8de32abe373f4a73, scope.project.id=676e986ff972410d9f498800d313476f, user.id=9243ebe24ae84157b74696c29bda3279) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:38.044370 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d0dbb-efe0-45b3-b0cc-021f67501059 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:38.045786 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d0dbb-efe0-45b3-b0cc-021f67501059 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:38.051325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 487/964] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Oct 19 02:15:37 2022] GET /identity/v3/role_assignments?user.id=9243ebe24ae84157b74696c29bda3279&scope.project.id=676e986ff972410d9f498800d313476f&role.id=d5070bb151d343ca8de32abe373f4a73 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:40.178011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074e827-3f3f-4880-9a33-ba4f88f30234 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:40.178520 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074e827-3f3f-4880-9a33-ba4f88f30234 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:40.178612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074e827-3f3f-4880-9a33-ba4f88f30234 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:40.179900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 478/965] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:15:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:15:40.184990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaca0d0-5146-40e3-94c8-866060e1f462 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:40.185405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaca0d0-5146-40e3-94c8-866060e1f462 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:40.185520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaca0d0-5146-40e3-94c8-866060e1f462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:40.203225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-adaca0d0-5146-40e3-94c8-866060e1f462 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:15:40.204399 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-adaca0d0-5146-40e3-94c8-866060e1f462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:15:40.268905 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 488/966] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:15:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:15:40.534476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074e827-3f3f-4880-9a33-ba4f88f30234 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:40.564722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:40.632191 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074e827-3f3f-4880-9a33-ba4f88f30234 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:40.663829 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4ca6-3e1a-4339-9967-d6c3c85c8cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:40.664444 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4ca6-3e1a-4339-9967-d6c3c85c8cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:40.664719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4ca6-3e1a-4339-9967-d6c3c85c8cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:40.664989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4ca6-3e1a-4339-9967-d6c3c85c8cc3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:40.665561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a4ca6-3e1a-4339-9967-d6c3c85c8cc3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:40.666925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a4ca6-3e1a-4339-9967-d6c3c85c8cc3 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-6e8a4ca6-3e1a-4339-9967-d6c3c85c8cc3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:40.675414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:40.676654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:40.678015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 479/967] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 19 02:15:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:40.682942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaca0d0-5146-40e3-94c8-866060e1f462 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:40.770113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:40.779225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaca0d0-5146-40e3-94c8-866060e1f462 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:40.813390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd668c7-a868-46f8-b88f-73750f34a1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:40.813926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd668c7-a868-46f8-b88f-73750f34a1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:40.814311 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd668c7-a868-46f8-b88f-73750f34a1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:40.814584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd668c7-a868-46f8-b88f-73750f34a1e7 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:40.815207 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd668c7-a868-46f8-b88f-73750f34a1e7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:40.815507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd668c7-a868-46f8-b88f-73750f34a1e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:40.816855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd668c7-a868-46f8-b88f-73750f34a1e7 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:40.822126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 489/968] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Oct 19 02:15:40 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:40.827909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4ca6-3e1a-4339-9967-d6c3c85c8cc3 None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:40.919818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:40.925327 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4ca6-3e1a-4339-9967-d6c3c85c8cc3 None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:40.957620 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7972e9-ec8a-42b5-b047-e997102a4083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:40.958250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7972e9-ec8a-42b5-b047-e997102a4083 None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:40.958533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7972e9-ec8a-42b5-b047-e997102a4083 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:40.958806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7972e9-ec8a-42b5-b047-e997102a4083 None admin] PATH_INFO: `/v3/services` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:40.959439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7972e9-ec8a-42b5-b047-e997102a4083 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:40.959739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7972e9-ec8a-42b5-b047-e997102a4083 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:40.961096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7972e9-ec8a-42b5-b047-e997102a4083 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:40.965304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 480/969] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Oct 19 02:15:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:43.075256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7875902-aec3-40ff-97f9-d5b06d324ce9 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:43.075788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7875902-aec3-40ff-97f9-d5b06d324ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:43.075788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7875902-aec3-40ff-97f9-d5b06d324ce9 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:43.077108 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 490/970] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:15:43.082244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d3078-522e-4585-9be9-6c35e250a665 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:43.082649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d3078-522e-4585-9be9-6c35e250a665 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:43.082785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d3078-522e-4585-9be9-6c35e250a665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:43.101276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-8c4d3078-522e-4585-9be9-6c35e250a665 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:15:43.102482 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8c4d3078-522e-4585-9be9-6c35e250a665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:15:43.167577 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 481/971] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:15:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:15:43.433138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7875902-aec3-40ff-97f9-d5b06d324ce9 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:43.540043 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7875902-aec3-40ff-97f9-d5b06d324ce9 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:43.572394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f953d-1e00-4475-86ce-94ebf7d72e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:43.572984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f953d-1e00-4475-86ce-94ebf7d72e28 None admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:43.573256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f953d-1e00-4475-86ce-94ebf7d72e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:43.573521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f953d-1e00-4475-86ce-94ebf7d72e28 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:43.574180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f953d-1e00-4475-86ce-94ebf7d72e28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:43.575534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f953d-1e00-4475-86ce-94ebf7d72e28 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:43.650534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 491/972] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Oct 19 02:15:43 2022] POST /identity/v3/services => generated 246 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:15:45.776886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7c56e-510e-413b-abcc-ee05e44a1dc8 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:45.777441 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7c56e-510e-413b-abcc-ee05e44a1dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:45.777441 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7c56e-510e-413b-abcc-ee05e44a1dc8 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:45.778905 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 482/973] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:15:45.784510 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c133-5c7a-454e-9203-723e7aa6f9d9 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:45.795289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c133-5c7a-454e-9203-723e7aa6f9d9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:45.795289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21c133-5c7a-454e-9203-723e7aa6f9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:45.803197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-2f21c133-5c7a-454e-9203-723e7aa6f9d9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:15:45.804409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2f21c133-5c7a-454e-9203-723e7aa6f9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:15:45.813397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:45.876210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 492/974] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:15:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 19 02:15:46.141834 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7c56e-510e-413b-abcc-ee05e44a1dc8 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:46.231771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:46.241995 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7c56e-510e-413b-abcc-ee05e44a1dc8 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:46.274971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519c4ee-2a77-4591-b078-43eac9ca303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:46.275584 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c4ee-2a77-4591-b078-43eac9ca303c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:46.275861 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c4ee-2a77-4591-b078-43eac9ca303c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:46.276130 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519c4ee-2a77-4591-b078-43eac9ca303c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:46.276705 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519c4ee-2a77-4591-b078-43eac9ca303c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:46.278064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519c4ee-2a77-4591-b078-43eac9ca303c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-2519c4ee-2a77-4591-b078-43eac9ca303c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:46.286600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:46.287613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:46.289261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 483/975] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 19 02:15:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:46.293738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21c133-5c7a-454e-9203-723e7aa6f9d9 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:46.388116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21c133-5c7a-454e-9203-723e7aa6f9d9 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:46.421460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc73cc8-700e-44a4-9dd6-0b1b577fe830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:46.422064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc73cc8-700e-44a4-9dd6-0b1b577fe830 None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:46.422356 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc73cc8-700e-44a4-9dd6-0b1b577fe830 None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:46.422623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc73cc8-700e-44a4-9dd6-0b1b577fe830 None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:46.423235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc73cc8-700e-44a4-9dd6-0b1b577fe830 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:46.423530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc73cc8-700e-44a4-9dd6-0b1b577fe830 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:46.424875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc73cc8-700e-44a4-9dd6-0b1b577fe830 None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:46.429785 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 493/976] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Oct 19 02:15:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:46.436005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519c4ee-2a77-4591-b078-43eac9ca303c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:46.532473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519c4ee-2a77-4591-b078-43eac9ca303c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:46.564916 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494000fd-4686-440b-a58a-10c2a091937a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:46.565509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494000fd-4686-440b-a58a-10c2a091937a None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:46.565787 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494000fd-4686-440b-a58a-10c2a091937a None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:46.566070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494000fd-4686-440b-a58a-10c2a091937a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:46.566779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494000fd-4686-440b-a58a-10c2a091937a None admin] RBAC: Adding query filter params (interface=public, service_id=108570ceb217417882f7ed1a28bf253f, region_id=RegionOne) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:46.567085 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494000fd-4686-440b-a58a-10c2a091937a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:46.569023 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494000fd-4686-440b-a58a-10c2a091937a None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:46.576275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 484/977] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Oct 19 02:15:46 2022] GET /identity/v3/endpoints?service_id=108570ceb217417882f7ed1a28bf253f&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:46.582123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc73cc8-700e-44a4-9dd6-0b1b577fe830 None admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:46.590848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:46.678454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc73cc8-700e-44a4-9dd6-0b1b577fe830 None admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:46.709943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb03ec-0b5c-48c6-93ff-76dbe2fe93db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:46.710595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb03ec-0b5c-48c6-93ff-76dbe2fe93db None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:46.710872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb03ec-0b5c-48c6-93ff-76dbe2fe93db None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:46.711142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb03ec-0b5c-48c6-93ff-76dbe2fe93db None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:46.711734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb03ec-0b5c-48c6-93ff-76dbe2fe93db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:46.713081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb03ec-0b5c-48c6-93ff-76dbe2fe93db None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:46.719576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 494/978] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Oct 19 02:15:46 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:15:48.842730 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f262f-9280-4fcc-99b2-414b86ab7c68 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:48.843288 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f262f-9280-4fcc-99b2-414b86ab7c68 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:48.843288 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f262f-9280-4fcc-99b2-414b86ab7c68 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:48.844700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 485/979] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:15:48.850468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d52228-4673-476b-9596-5640600c81f4 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:48.850901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d52228-4673-476b-9596-5640600c81f4 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:48.850995 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d52228-4673-476b-9596-5640600c81f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:48.869200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c1d52228-4673-476b-9596-5640600c81f4 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:15:48.870386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c1d52228-4673-476b-9596-5640600c81f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:15:48.929308 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:48.934619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 495/980] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:15:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 19 02:15:49.204186 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f262f-9280-4fcc-99b2-414b86ab7c68 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:49.223895 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:49.303205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f262f-9280-4fcc-99b2-414b86ab7c68 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:49.335844 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20c8d8-0bff-4db0-b3a9-394cf239839c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:49.336463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20c8d8-0bff-4db0-b3a9-394cf239839c None admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:49.336740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20c8d8-0bff-4db0-b3a9-394cf239839c None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:49.337011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20c8d8-0bff-4db0-b3a9-394cf239839c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:49.337595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20c8d8-0bff-4db0-b3a9-394cf239839c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:49.338975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20c8d8-0bff-4db0-b3a9-394cf239839c None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application [None req-8d20c8d8-0bff-4db0-b3a9-394cf239839c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 19 02:15:49.347625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 19 02:15:49.349017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.get_or_create( Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application with Lock( Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self._enter() Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application return self.creator() Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application created_value = creator( Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: ERROR keystone.server.flask.application Oct 19 02:15:49.350196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 486/981] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Oct 19 02:15:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 19 02:15:49.355266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d52228-4673-476b-9596-5640600c81f4 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:49.449396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d52228-4673-476b-9596-5640600c81f4 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:49.482351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d86a7-e019-4e34-adc2-2e092bab810b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:49.482935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d86a7-e019-4e34-adc2-2e092bab810b None admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:49.483208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d86a7-e019-4e34-adc2-2e092bab810b None admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:49.483475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d86a7-e019-4e34-adc2-2e092bab810b None admin] PATH_INFO: `/v3/services` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:49.484083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d86a7-e019-4e34-adc2-2e092bab810b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:15:49.484382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d86a7-e019-4e34-adc2-2e092bab810b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:49.485743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d86a7-e019-4e34-adc2-2e092bab810b None admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:49.491112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 496/982] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Oct 19 02:15:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:15:49.496996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20c8d8-0bff-4db0-b3a9-394cf239839c None admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:15:49.585571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:49.602826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20c8d8-0bff-4db0-b3a9-394cf239839c None admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:15:49.635022 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7d6a4-d280-434b-a53d-3d2315ff0000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:15:49.635557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7d6a4-d280-434b-a53d-3d2315ff0000 None admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:15:49.635832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7d6a4-d280-434b-a53d-3d2315ff0000 None admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:15:49.636096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7d6a4-d280-434b-a53d-3d2315ff0000 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:15:49.636747 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7d6a4-d280-434b-a53d-3d2315ff0000 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:15:49.638119 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7d6a4-d280-434b-a53d-3d2315ff0000 None admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:15:49.761268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 487/983] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:15:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:15:59.487782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=69 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:15:59.755940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:23:42.666715 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050d84b-52c0-4b40-931e-ec68d54dae29 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:42.667279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050d84b-52c0-4b40-931e-ec68d54dae29 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:42.667362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050d84b-52c0-4b40-931e-ec68d54dae29 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:42.668634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 497/984] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:23:42.674494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d34ee1-ab8b-4ab6-8ed8-9aa934417623 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:42.674980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d34ee1-ab8b-4ab6-8ed8-9aa934417623 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:42.675067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d34ee1-ab8b-4ab6-8ed8-9aa934417623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:42.768942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-33d34ee1-ab8b-4ab6-8ed8-9aa934417623 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:23:42.770157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-33d34ee1-ab8b-4ab6-8ed8-9aa934417623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:42.878934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 488/985] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:23:43.172904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764a4d1-db8a-434e-a484-918447b092e9 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:43.173252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764a4d1-db8a-434e-a484-918447b092e9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:43.173449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764a4d1-db8a-434e-a484-918447b092e9 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:43.175234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 498/986] 10.222.0.20 () {54 vars in 964 bytes} [Wed Oct 19 02:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:23:43.181507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc4d63-a3da-43ab-a552-f30ac34d4019 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:43.182037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc4d63-a3da-43ab-a552-f30ac34d4019 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:43.182358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc4d63-a3da-43ab-a552-f30ac34d4019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:43.269514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-cdcc4d63-a3da-43ab-a552-f30ac34d4019 None None] MFA Rules not processed for user `b5ff8583997843258e315413abac172f`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:23:43.270752 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cdcc4d63-a3da-43ab-a552-f30ac34d4019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:43.292231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:23:43.349489 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 489/987] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Oct 19 02:23:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:23:43.359345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764a4d1-db8a-434e-a484-918447b092e9 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:23:43.361138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c764a4d1-db8a-434e-a484-918447b092e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:43.476707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764a4d1-db8a-434e-a484-918447b092e9 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:23:43.558818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44b450-f7d4-4385-ab43-0232a2579a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:23:43.559544 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44b450-f7d4-4385-ab43-0232a2579a8a service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:43.560013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44b450-f7d4-4385-ab43-0232a2579a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:43.560361 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44b450-f7d4-4385-ab43-0232a2579a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:43.562661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5f44b450-f7d4-4385-ab43-0232a2579a8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:43.624531 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:23:43.689678 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44b450-f7d4-4385-ab43-0232a2579a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:23:43.691271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f44b450-f7d4-4385-ab43-0232a2579a8a service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:23:43.714891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 499/988] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:23:46.233782 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d549345-e8c3-49a8-ad54-7a5934f95f33 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:46.234312 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d549345-e8c3-49a8-ad54-7a5934f95f33 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:46.234312 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d549345-e8c3-49a8-ad54-7a5934f95f33 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:46.235613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 490/989] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:23:46.240804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf0c5d-9001-47fe-901c-ce8943eeff10 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:46.241224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf0c5d-9001-47fe-901c-ce8943eeff10 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:46.241341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf0c5d-9001-47fe-901c-ce8943eeff10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:46.309860 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-3adf0c5d-9001-47fe-901c-ce8943eeff10 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:23:46.311647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3adf0c5d-9001-47fe-901c-ce8943eeff10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:46.405391 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 500/990] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:23:46.684469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d549345-e8c3-49a8-ad54-7a5934f95f33 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:23:46.686565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8d549345-e8c3-49a8-ad54-7a5934f95f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:46.786419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d549345-e8c3-49a8-ad54-7a5934f95f33 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:23:46.856406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:23:46.859175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9a722-13e1-41a5-b3c4-eb450cd08339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:23:46.859776 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9a722-13e1-41a5-b3c4-eb450cd08339 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:46.860051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9a722-13e1-41a5-b3c4-eb450cd08339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:46.860320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9a722-13e1-41a5-b3c4-eb450cd08339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:46.861887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8ab9a722-13e1-41a5-b3c4-eb450cd08339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:46.973822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9a722-13e1-41a5-b3c4-eb450cd08339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:23:46.975191 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab9a722-13e1-41a5-b3c4-eb450cd08339 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:23:46.993234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 491/991] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:23:49.602500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e48f9b-3efa-484c-afb3-b7d874348414 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:49.603278 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e48f9b-3efa-484c-afb3-b7d874348414 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:49.603278 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e48f9b-3efa-484c-afb3-b7d874348414 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:49.604645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 501/992] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:23:49.609803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c1f3-f4ea-428d-8a10-135e0f2338a2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:49.610234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c1f3-f4ea-428d-8a10-135e0f2338a2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:49.610375 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c1f3-f4ea-428d-8a10-135e0f2338a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:49.673749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-72a8c1f3-f4ea-428d-8a10-135e0f2338a2 None None] MFA Rules not processed for user `98f9702915254d03a5ddc76409a170e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:23:49.674979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-72a8c1f3-f4ea-428d-8a10-135e0f2338a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:49.725955 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:23:49.769124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 492/993] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:23:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:23:50.050341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e48f9b-3efa-484c-afb3-b7d874348414 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:23:50.051884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-43e48f9b-3efa-484c-afb3-b7d874348414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:50.062209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:23:50.153083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e48f9b-3efa-484c-afb3-b7d874348414 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:23:50.223322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330578c9-46a0-4086-b378-b3344f9c6a70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:23:50.223917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330578c9-46a0-4086-b378-b3344f9c6a70 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:50.224198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330578c9-46a0-4086-b378-b3344f9c6a70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:50.224467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330578c9-46a0-4086-b378-b3344f9c6a70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:50.226024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-330578c9-46a0-4086-b378-b3344f9c6a70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:50.285727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:23:50.331598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330578c9-46a0-4086-b378-b3344f9c6a70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:23:50.333019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330578c9-46a0-4086-b378-b3344f9c6a70 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:23:50.350651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 502/994] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:23:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:23:53.301875 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19602e-d87e-4658-a469-0eb06444dd44 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:53.302443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19602e-d87e-4658-a469-0eb06444dd44 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:53.302443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19602e-d87e-4658-a469-0eb06444dd44 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:53.303799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 493/995] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:23:53.309151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb27125-3820-453a-87c9-e0cf8b775d5b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:53.309524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb27125-3820-453a-87c9-e0cf8b775d5b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:53.309690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb27125-3820-453a-87c9-e0cf8b775d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:53.374233 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-7fb27125-3820-453a-87c9-e0cf8b775d5b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:23:53.375327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7fb27125-3820-453a-87c9-e0cf8b775d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:53.468345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 503/996] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:23:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:23:53.765100 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22e5a2-7566-436a-9d3a-c1fb6aeae86d None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:53.765380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22e5a2-7566-436a-9d3a-c1fb6aeae86d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:53.765653 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22e5a2-7566-436a-9d3a-c1fb6aeae86d None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:53.767035 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 494/997] 10.222.0.20 () {54 vars in 964 bytes} [Wed Oct 19 02:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:23:53.772702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f2769-7841-4333-8a9a-8ae07d972d48 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:53.772988 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f2769-7841-4333-8a9a-8ae07d972d48 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:53.773263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f2769-7841-4333-8a9a-8ae07d972d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:53.795993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:23:53.849751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-0d4f2769-7841-4333-8a9a-8ae07d972d48 None None] MFA Rules not processed for user `b5ff8583997843258e315413abac172f`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:23:53.850959 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0d4f2769-7841-4333-8a9a-8ae07d972d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:53.923794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 504/998] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Oct 19 02:23:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:23:53.932813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22e5a2-7566-436a-9d3a-c1fb6aeae86d None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:23:53.934368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cd22e5a2-7566-436a-9d3a-c1fb6aeae86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:54.035301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22e5a2-7566-436a-9d3a-c1fb6aeae86d None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:23:54.107273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05caae8-483e-4260-9ad2-b678152edfa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:23:54.107896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05caae8-483e-4260-9ad2-b678152edfa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:54.108147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05caae8-483e-4260-9ad2-b678152edfa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:54.108412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05caae8-483e-4260-9ad2-b678152edfa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:54.109976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d05caae8-483e-4260-9ad2-b678152edfa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:54.116761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:23:54.216057 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05caae8-483e-4260-9ad2-b678152edfa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:23:54.217537 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05caae8-483e-4260-9ad2-b678152edfa7 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:23:54.235091 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 495/999] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:23:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:23:57.823123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311a8fe-f18e-45b2-8ef3-ef59beac28a3 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:57.823658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311a8fe-f18e-45b2-8ef3-ef59beac28a3 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:57.823658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311a8fe-f18e-45b2-8ef3-ef59beac28a3 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:57.825176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 505/1000] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:23:57.831103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f095b1-ad61-4a83-a255-72b4e8876e5e None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:57.831517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f095b1-ad61-4a83-a255-72b4e8876e5e None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:57.831646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f095b1-ad61-4a83-a255-72b4e8876e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:57.899588 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-91f095b1-ad61-4a83-a255-72b4e8876e5e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:23:57.900778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-91f095b1-ad61-4a83-a255-72b4e8876e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:57.961655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:23:58.004736 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 496/1001] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:23:58.286630 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311a8fe-f18e-45b2-8ef3-ef59beac28a3 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:23:58.288182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6311a8fe-f18e-45b2-8ef3-ef59beac28a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:58.391375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311a8fe-f18e-45b2-8ef3-ef59beac28a3 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:23:58.404175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:23:58.464833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16557844-2bbe-4c54-8a15-d87d0e39fdda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:23:58.465426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16557844-2bbe-4c54-8a15-d87d0e39fdda service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:23:58.465698 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16557844-2bbe-4c54-8a15-d87d0e39fdda service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:23:58.465968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16557844-2bbe-4c54-8a15-d87d0e39fdda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:23:58.468168 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-16557844-2bbe-4c54-8a15-d87d0e39fdda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:23:58.572673 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16557844-2bbe-4c54-8a15-d87d0e39fdda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:23:58.574139 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16557844-2bbe-4c54-8a15-d87d0e39fdda service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:23:58.594141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 506/1002] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:00.772633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f390e-4ca8-45be-a7a4-ae900e12ec79 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:00.773178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f390e-4ca8-45be-a7a4-ae900e12ec79 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:00.773178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f390e-4ca8-45be-a7a4-ae900e12ec79 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:00.774534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 497/1003] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:00.779656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c253dcf-1512-419b-a241-a19bda93c239 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:00.780572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c253dcf-1512-419b-a241-a19bda93c239 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:00.780714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c253dcf-1512-419b-a241-a19bda93c239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:00.834535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:00.854241 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-0c253dcf-1512-419b-a241-a19bda93c239 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:00.855429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0c253dcf-1512-419b-a241-a19bda93c239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:00.954027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 507/1004] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:01.236640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f390e-4ca8-45be-a7a4-ae900e12ec79 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:01.238252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9c9f390e-4ca8-45be-a7a4-ae900e12ec79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:01.338656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f390e-4ca8-45be-a7a4-ae900e12ec79 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:01.410315 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0f75df-7a5b-4566-aad8-78da2a653766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:01.410907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f75df-7a5b-4566-aad8-78da2a653766 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:01.411180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f75df-7a5b-4566-aad8-78da2a653766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:01.411446 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0f75df-7a5b-4566-aad8-78da2a653766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:01.412996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1b0f75df-7a5b-4566-aad8-78da2a653766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:01.429527 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:01.520270 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f75df-7a5b-4566-aad8-78da2a653766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:01.521700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0f75df-7a5b-4566-aad8-78da2a653766 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:01.539526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 498/1005] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:03.754209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7578c9a-6d65-479d-b270-1402cd510812 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:03.754764 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7578c9a-6d65-479d-b270-1402cd510812 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:03.754764 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7578c9a-6d65-479d-b270-1402cd510812 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:03.756081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 508/1006] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:03.761485 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae313-d4db-418b-89bf-19b00ab83799 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:03.761914 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae313-d4db-418b-89bf-19b00ab83799 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:03.762036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eae313-d4db-418b-89bf-19b00ab83799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:03.828730 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-a8eae313-d4db-418b-89bf-19b00ab83799 None None] MFA Rules not processed for user `98f9702915254d03a5ddc76409a170e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:03.829908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a8eae313-d4db-418b-89bf-19b00ab83799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:03.894550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:03.926566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 499/1007] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:04.207521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7578c9a-6d65-479d-b270-1402cd510812 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:04.209606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a7578c9a-6d65-479d-b270-1402cd510812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:04.328004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7578c9a-6d65-479d-b270-1402cd510812 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:04.349755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:04.407377 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4ee14-0004-4fe2-8fd7-cf681a542e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:04.408017 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ee14-0004-4fe2-8fd7-cf681a542e40 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:04.408291 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ee14-0004-4fe2-8fd7-cf681a542e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:04.408558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4ee14-0004-4fe2-8fd7-cf681a542e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:04.410261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-05e4ee14-0004-4fe2-8fd7-cf681a542e40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:04.528473 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4ee14-0004-4fe2-8fd7-cf681a542e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:04.530604 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4ee14-0004-4fe2-8fd7-cf681a542e40 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:04.548634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 509/1008] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:07.070400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c138d-dfeb-4281-bc76-c84931700341 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:07.070931 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c138d-dfeb-4281-bc76-c84931700341 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:07.070931 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c138d-dfeb-4281-bc76-c84931700341 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:07.072270 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 500/1009] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:07.078126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f7a15-e470-4b83-a702-252e6a2af331 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:07.078521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f7a15-e470-4b83-a702-252e6a2af331 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:07.078668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f7a15-e470-4b83-a702-252e6a2af331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:07.144911 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:07.148149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-aa9f7a15-e470-4b83-a702-252e6a2af331 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:07.149323 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-aa9f7a15-e470-4b83-a702-252e6a2af331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:07.243096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 510/1010] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:07.522846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c138d-dfeb-4281-bc76-c84931700341 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:07.524372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9f9c138d-dfeb-4281-bc76-c84931700341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:07.625784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c138d-dfeb-4281-bc76-c84931700341 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:07.696806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276d387-935f-463a-b1d3-34e655293376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:07.697410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276d387-935f-463a-b1d3-34e655293376 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:07.697689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276d387-935f-463a-b1d3-34e655293376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:07.697960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276d387-935f-463a-b1d3-34e655293376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:07.699563 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8276d387-935f-463a-b1d3-34e655293376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:07.725885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:07.805792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276d387-935f-463a-b1d3-34e655293376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:07.807175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276d387-935f-463a-b1d3-34e655293376 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:07.825063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 501/1011] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:10.001175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdc4b9-c062-4510-b475-5e3886768470 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:10.001727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdc4b9-c062-4510-b475-5e3886768470 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:10.001727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccdc4b9-c062-4510-b475-5e3886768470 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:10.003065 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 511/1012] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:10.008210 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74781ef2-512f-471e-9e84-7ec8e54d3cef None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:10.008619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74781ef2-512f-471e-9e84-7ec8e54d3cef None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:10.008755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74781ef2-512f-471e-9e84-7ec8e54d3cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:10.073490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-74781ef2-512f-471e-9e84-7ec8e54d3cef None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:10.074707 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-74781ef2-512f-471e-9e84-7ec8e54d3cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:10.147571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:10.168656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 502/1013] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:24:10.449774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdc4b9-c062-4510-b475-5e3886768470 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:10.451380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4ccdc4b9-c062-4510-b475-5e3886768470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:10.552182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccdc4b9-c062-4510-b475-5e3886768470 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:10.579878 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:10.624835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516df64d-ce1f-4172-aca3-1e55b9a7bce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:10.625423 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df64d-ce1f-4172-aca3-1e55b9a7bce5 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:10.625696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df64d-ce1f-4172-aca3-1e55b9a7bce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:10.625968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df64d-ce1f-4172-aca3-1e55b9a7bce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:10.627535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-516df64d-ce1f-4172-aca3-1e55b9a7bce5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:10.731755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516df64d-ce1f-4172-aca3-1e55b9a7bce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:10.733152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516df64d-ce1f-4172-aca3-1e55b9a7bce5 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:10.750648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 512/1014] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:13.541438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641e5b7-5a4c-45b3-a378-0d3173f5f727 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:13.541986 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641e5b7-5a4c-45b3-a378-0d3173f5f727 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:13.541986 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641e5b7-5a4c-45b3-a378-0d3173f5f727 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:13.543351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 503/1015] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:13.549008 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51730b00-d546-418a-93a0-ac09486f864d None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:13.549397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51730b00-d546-418a-93a0-ac09486f864d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:13.549555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51730b00-d546-418a-93a0-ac09486f864d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:13.614949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-51730b00-d546-418a-93a0-ac09486f864d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:13.616166 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-51730b00-d546-418a-93a0-ac09486f864d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:13.624232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:13.707868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 513/1016] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:13.993208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641e5b7-5a4c-45b3-a378-0d3173f5f727 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:13.994750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5641e5b7-5a4c-45b3-a378-0d3173f5f727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:14.094576 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641e5b7-5a4c-45b3-a378-0d3173f5f727 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:14.166417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e172f-bb02-4780-902f-534c90a66089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:14.167014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e172f-bb02-4780-902f-534c90a66089 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:14.167288 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e172f-bb02-4780-902f-534c90a66089 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:14.167553 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e172f-bb02-4780-902f-534c90a66089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:14.169117 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-906e172f-bb02-4780-902f-534c90a66089 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:14.207485 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:14.277046 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e172f-bb02-4780-902f-534c90a66089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:14.280082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e172f-bb02-4780-902f-534c90a66089 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:14.296885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 504/1017] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:18.678926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fd534-4bd0-4956-bb09-6f358caf8d72 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:18.679355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fd534-4bd0-4956-bb09-6f358caf8d72 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:18.679456 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fd534-4bd0-4956-bb09-6f358caf8d72 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:18.680789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 514/1018] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:18.686587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb843f-7854-47ae-b8d1-3eee57a68bb8 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:18.686998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb843f-7854-47ae-b8d1-3eee57a68bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:18.687141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb843f-7854-47ae-b8d1-3eee57a68bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:18.754773 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-53bb843f-7854-47ae-b8d1-3eee57a68bb8 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:18.755968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-53bb843f-7854-47ae-b8d1-3eee57a68bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:18.840791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:18.852204 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 505/1019] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:19.133333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fd534-4bd0-4956-bb09-6f358caf8d72 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:19.134913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3e9fd534-4bd0-4956-bb09-6f358caf8d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:19.236552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fd534-4bd0-4956-bb09-6f358caf8d72 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:19.273156 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:19.308421 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663cee2-0b42-4b87-8829-b4aa79983f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:19.309613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663cee2-0b42-4b87-8829-b4aa79983f14 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:19.309893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663cee2-0b42-4b87-8829-b4aa79983f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:19.310195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663cee2-0b42-4b87-8829-b4aa79983f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:19.311783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6663cee2-0b42-4b87-8829-b4aa79983f14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:19.418239 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663cee2-0b42-4b87-8829-b4aa79983f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:19.419585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663cee2-0b42-4b87-8829-b4aa79983f14 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:19.437537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 515/1020] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:22.553343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8d0f3-8e43-48d9-a9d2-13d8a19af5f0 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:22.553932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8d0f3-8e43-48d9-a9d2-13d8a19af5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:22.553932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8d0f3-8e43-48d9-a9d2-13d8a19af5f0 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:22.555272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 506/1021] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:22.560448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425f7e7-2812-4c36-bcdd-cf648d9204aa None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:22.560890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425f7e7-2812-4c36-bcdd-cf648d9204aa None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:22.561663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e425f7e7-2812-4c36-bcdd-cf648d9204aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:22.627478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e425f7e7-2812-4c36-bcdd-cf648d9204aa None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:22.628655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e425f7e7-2812-4c36-bcdd-cf648d9204aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:22.646737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:22.728356 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 516/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:23.017190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8d0f3-8e43-48d9-a9d2-13d8a19af5f0 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:23.018794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d5a8d0f3-8e43-48d9-a9d2-13d8a19af5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:23.130601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8d0f3-8e43-48d9-a9d2-13d8a19af5f0 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:23.204652 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae29bf-7114-4e9c-a235-07d51a913547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:23.205226 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae29bf-7114-4e9c-a235-07d51a913547 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:23.205563 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae29bf-7114-4e9c-a235-07d51a913547 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:23.205999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae29bf-7114-4e9c-a235-07d51a913547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:23.207725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7bae29bf-7114-4e9c-a235-07d51a913547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:23.250674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:23.316844 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae29bf-7114-4e9c-a235-07d51a913547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:23.318304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae29bf-7114-4e9c-a235-07d51a913547 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:23.337125 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 507/1023] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:27.760611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7454d8-9cbf-46e9-9c89-f0b19dcf1f13 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:27.761143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7454d8-9cbf-46e9-9c89-f0b19dcf1f13 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:27.761143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7454d8-9cbf-46e9-9c89-f0b19dcf1f13 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:27.762551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 517/1024] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:27.768749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b0496-6101-4d06-a8c9-5bf49e0e5e0f None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:27.769191 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b0496-6101-4d06-a8c9-5bf49e0e5e0f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:27.769307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b0496-6101-4d06-a8c9-5bf49e0e5e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:27.840896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-595b0496-6101-4d06-a8c9-5bf49e0e5e0f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:27.842033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-595b0496-6101-4d06-a8c9-5bf49e0e5e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:27.941357 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:27.946101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 508/1025] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:28.226804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7454d8-9cbf-46e9-9c89-f0b19dcf1f13 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:28.228419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cf7454d8-9cbf-46e9-9c89-f0b19dcf1f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:28.331592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7454d8-9cbf-46e9-9c89-f0b19dcf1f13 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:28.379002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:28.404244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5eec42-b628-424f-8c1b-fb6aa45b78a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:28.404878 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5eec42-b628-424f-8c1b-fb6aa45b78a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:28.405153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5eec42-b628-424f-8c1b-fb6aa45b78a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:28.405422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5eec42-b628-424f-8c1b-fb6aa45b78a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:28.407076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9c5eec42-b628-424f-8c1b-fb6aa45b78a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:28.516086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5eec42-b628-424f-8c1b-fb6aa45b78a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:28.517525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5eec42-b628-424f-8c1b-fb6aa45b78a6 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:28.535329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 518/1026] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:33.396608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d645d80-088a-44f6-a072-dc109d37d3d2 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:33.396608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d645d80-088a-44f6-a072-dc109d37d3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:33.396608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d645d80-088a-44f6-a072-dc109d37d3d2 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:33.396608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 509/1027] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:24:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:33.400701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8791618-99c1-4ea6-8e26-28c8d4b7fc81 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:33.400701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8791618-99c1-4ea6-8e26-28c8d4b7fc81 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:33.400701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8791618-99c1-4ea6-8e26-28c8d4b7fc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:33.490905 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e8791618-99c1-4ea6-8e26-28c8d4b7fc81 None None] MFA Rules not processed for user `9243ebe24ae84157b74696c29bda3279`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:33.500314 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e8791618-99c1-4ea6-8e26-28c8d4b7fc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:33.517447 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:33.629573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 519/1028] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:24:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:38.089059 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710af54-8217-4a73-85a1-de2d22a7c961 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:38.090146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710af54-8217-4a73-85a1-de2d22a7c961 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:38.090479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710af54-8217-4a73-85a1-de2d22a7c961 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:38.091940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 510/1029] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:24:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:38.098011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601afa1b-de98-4685-a03b-b28d320c1539 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:38.098838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601afa1b-de98-4685-a03b-b28d320c1539 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:38.099248 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601afa1b-de98-4685-a03b-b28d320c1539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:38.179510 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-601afa1b-de98-4685-a03b-b28d320c1539 None None] MFA Rules not processed for user `9243ebe24ae84157b74696c29bda3279`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:38.180780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-601afa1b-de98-4685-a03b-b28d320c1539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:38.258393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:38.298163 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 520/1030] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:24:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:39.521431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac394b9-5739-4e2b-a1a6-3694a9e6dc62 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:39.521967 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac394b9-5739-4e2b-a1a6-3694a9e6dc62 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:39.521967 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac394b9-5739-4e2b-a1a6-3694a9e6dc62 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:39.523319 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 511/1031] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:24:39.528303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7857-7403-466d-8478-f90aa8dba14f None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:39.528720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7857-7403-466d-8478-f90aa8dba14f None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:39.528954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a7857-7403-466d-8478-f90aa8dba14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:39.603977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-cb2a7857-7403-466d-8478-f90aa8dba14f None None] MFA Rules not processed for user `9243ebe24ae84157b74696c29bda3279`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:39.605498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cb2a7857-7403-466d-8478-f90aa8dba14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:39.709774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 521/1032] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:24:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:24:43.215219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448ab6-2887-4dba-a566-2914e0784eee None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:43.216108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448ab6-2887-4dba-a566-2914e0784eee None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:43.216393 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448ab6-2887-4dba-a566-2914e0784eee None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:43.218429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 512/1033] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:43.223904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c4f1f-d10a-48c7-9cfa-982c86276f45 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:43.224354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c4f1f-d10a-48c7-9cfa-982c86276f45 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:43.224454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c4f1f-d10a-48c7-9cfa-982c86276f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:43.266928 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:43.300943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-840c4f1f-d10a-48c7-9cfa-982c86276f45 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:43.309171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-840c4f1f-d10a-48c7-9cfa-982c86276f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:43.409362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 522/1034] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:43.708809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00489-b106-4e89-867e-47fb990ae6e8 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:43.709494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00489-b106-4e89-867e-47fb990ae6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:43.709893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00489-b106-4e89-867e-47fb990ae6e8 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:43.712613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 513/1035] 10.222.0.20 () {54 vars in 961 bytes} [Wed Oct 19 02:24:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:43.720275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaea572-9ea9-487d-bb6b-7160bf77bcbd None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:43.720767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaea572-9ea9-487d-bb6b-7160bf77bcbd None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:43.726727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaea572-9ea9-487d-bb6b-7160bf77bcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:43.801985 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-deaea572-9ea9-487d-bb6b-7160bf77bcbd None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:43.803279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-deaea572-9ea9-487d-bb6b-7160bf77bcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:43.866573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:43.946066 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 523/1036] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Oct 19 02:24:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:43.954719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb00489-b106-4e89-867e-47fb990ae6e8 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:43.956503 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-edb00489-b106-4e89-867e-47fb990ae6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:44.118236 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb00489-b106-4e89-867e-47fb990ae6e8 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:44.198956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:44.253592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b89e3a-6aac-4e5a-b125-69b1378aaabc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:44.254496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b89e3a-6aac-4e5a-b125-69b1378aaabc service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:44.254871 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b89e3a-6aac-4e5a-b125-69b1378aaabc service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:44.255217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b89e3a-6aac-4e5a-b125-69b1378aaabc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:44.256920 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c6b89e3a-6aac-4e5a-b125-69b1378aaabc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:44.365657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b89e3a-6aac-4e5a-b125-69b1378aaabc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:44.367619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b89e3a-6aac-4e5a-b125-69b1378aaabc service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:44.388106 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 514/1037] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:46.770872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69064c-c2e8-4634-9d45-e9bb52438805 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:46.771296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69064c-c2e8-4634-9d45-e9bb52438805 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:46.771503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69064c-c2e8-4634-9d45-e9bb52438805 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:46.778806 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 524/1038] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:46.778990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b541567-72f6-4f22-b028-e453d176f19b None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:46.778990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b541567-72f6-4f22-b028-e453d176f19b None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:46.786285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b541567-72f6-4f22-b028-e453d176f19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:46.861805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-3b541567-72f6-4f22-b028-e453d176f19b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:46.863101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3b541567-72f6-4f22-b028-e453d176f19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:46.912808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:46.978128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 515/1039] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:47.261197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69064c-c2e8-4634-9d45-e9bb52438805 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:47.262912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ff69064c-c2e8-4634-9d45-e9bb52438805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:47.450194 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69064c-c2e8-4634-9d45-e9bb52438805 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:47.460428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:47.583324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd5e2f-f9da-43e1-8d02-dbe1c0b97f4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:47.583926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd5e2f-f9da-43e1-8d02-dbe1c0b97f4a service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:47.584204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd5e2f-f9da-43e1-8d02-dbe1c0b97f4a service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:47.584471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd5e2f-f9da-43e1-8d02-dbe1c0b97f4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:47.586070 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e6fd5e2f-f9da-43e1-8d02-dbe1c0b97f4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:47.679305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd5e2f-f9da-43e1-8d02-dbe1c0b97f4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:47.680946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd5e2f-f9da-43e1-8d02-dbe1c0b97f4a service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:47.699419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 525/1040] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:50.359829 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7e34-6879-4f45-819d-4368e1b6cef3 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:50.360374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7e34-6879-4f45-819d-4368e1b6cef3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:50.360374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f7e34-6879-4f45-819d-4368e1b6cef3 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:50.361752 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 516/1041] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:50.367255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eb3b5-4067-4043-8277-8744157c1464 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:50.367703 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eb3b5-4067-4043-8277-8744157c1464 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:50.367872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380eb3b5-4067-4043-8277-8744157c1464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:50.384939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:50.444093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-380eb3b5-4067-4043-8277-8744157c1464 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:50.445286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-380eb3b5-4067-4043-8277-8744157c1464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:50.548313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 526/1042] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:50.835854 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007a46a-963b-437a-be04-e78d34273b43 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:50.836144 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007a46a-963b-437a-be04-e78d34273b43 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:50.841474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007a46a-963b-437a-be04-e78d34273b43 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:50.841474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 517/1043] 10.222.0.20 () {54 vars in 961 bytes} [Wed Oct 19 02:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:50.843366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664762c7-3041-4f86-9cd8-aa7fa08ca901 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:50.843654 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664762c7-3041-4f86-9cd8-aa7fa08ca901 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:50.844000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664762c7-3041-4f86-9cd8-aa7fa08ca901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:50.923593 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-664762c7-3041-4f86-9cd8-aa7fa08ca901 None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:50.924799 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-664762c7-3041-4f86-9cd8-aa7fa08ca901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:50.960560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:51.046438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 527/1044] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Oct 19 02:24:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:51.054257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007a46a-963b-437a-be04-e78d34273b43 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:51.055994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c007a46a-963b-437a-be04-e78d34273b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:51.216214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007a46a-963b-437a-be04-e78d34273b43 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:51.239523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:51.343090 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0923c72-86e7-4773-9190-c11189326165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:51.343856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0923c72-86e7-4773-9190-c11189326165 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:51.344135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0923c72-86e7-4773-9190-c11189326165 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:51.344399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0923c72-86e7-4773-9190-c11189326165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:51.346079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c0923c72-86e7-4773-9190-c11189326165 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:51.444718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc7334-066a-4649-8ffc-54b9650c9aa1 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:51.445231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc7334-066a-4649-8ffc-54b9650c9aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:51.445231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc7334-066a-4649-8ffc-54b9650c9aa1 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:51.446737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 528/1045] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:24:51.454323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0923c72-86e7-4773-9190-c11189326165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:51.455469 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6934-5ab9-4946-bb45-df3fad293b61 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:51.455807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6934-5ab9-4946-bb45-df3fad293b61 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:51.456038 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0923c72-86e7-4773-9190-c11189326165 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:51.456151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb6934-5ab9-4946-bb45-df3fad293b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:51.474976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 518/1046] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:51.541584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-4dfb6934-5ab9-4946-bb45-df3fad293b61 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:51.542860 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4dfb6934-5ab9-4946-bb45-df3fad293b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:51.653009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:51.660149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 529/1047] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:52.020092 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db32c35-16c9-4a9f-b2e2-5ee5006f9452 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:52.020388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db32c35-16c9-4a9f-b2e2-5ee5006f9452 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:52.020666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db32c35-16c9-4a9f-b2e2-5ee5006f9452 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:52.022014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 519/1048] 10.222.0.20 () {54 vars in 963 bytes} [Wed Oct 19 02:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:52.033317 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a771ac-79cc-445b-b2a4-77cae69bbccc None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:52.033610 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a771ac-79cc-445b-b2a4-77cae69bbccc None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:52.033888 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a771ac-79cc-445b-b2a4-77cae69bbccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:52.137144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b0a771ac-79cc-445b-b2a4-77cae69bbccc None None] MFA Rules not processed for user `e63c5bd8c4264db697947f025c67f512`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:52.138508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b0a771ac-79cc-445b-b2a4-77cae69bbccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:52.230532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 530/1049] 10.222.0.20 () {58 vars in 1060 bytes} [Wed Oct 19 02:24:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:52.239076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db32c35-16c9-4a9f-b2e2-5ee5006f9452 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:52.240571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9db32c35-16c9-4a9f-b2e2-5ee5006f9452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:52.272921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:52.370888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db32c35-16c9-4a9f-b2e2-5ee5006f9452 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:52.450359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136d230-04c0-466a-9fb3-9d1bc1f289f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:52.450964 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d230-04c0-466a-9fb3-9d1bc1f289f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:52.451234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d230-04c0-466a-9fb3-9d1bc1f289f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:52.451499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d230-04c0-466a-9fb3-9d1bc1f289f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:52.453095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d136d230-04c0-466a-9fb3-9d1bc1f289f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:52.538665 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:52.572140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136d230-04c0-466a-9fb3-9d1bc1f289f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:52.574113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136d230-04c0-466a-9fb3-9d1bc1f289f0 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:52.594828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 520/1050] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:53.910285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702e2b-7776-443d-82a5-7c9bdec46b8b None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:53.910839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702e2b-7776-443d-82a5-7c9bdec46b8b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:53.910839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53702e2b-7776-443d-82a5-7c9bdec46b8b None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:53.912155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 531/1051] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:24:53.917493 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182de3-ef39-4c65-b8ea-50fb0dbdaabc None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:53.918238 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182de3-ef39-4c65-b8ea-50fb0dbdaabc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:53.918238 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182de3-ef39-4c65-b8ea-50fb0dbdaabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:53.983900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-94182de3-ef39-4c65-b8ea-50fb0dbdaabc None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:53.985102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-94182de3-ef39-4c65-b8ea-50fb0dbdaabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:54.083058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 521/1052] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:54.380271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53702e2b-7776-443d-82a5-7c9bdec46b8b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:54.381986 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-53702e2b-7776-443d-82a5-7c9bdec46b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:54.473492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:54.536728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53702e2b-7776-443d-82a5-7c9bdec46b8b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:54.652048 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fd665-f3d8-4989-9576-21381dc5ec0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:54.652651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fd665-f3d8-4989-9576-21381dc5ec0e service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:54.652918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fd665-f3d8-4989-9576-21381dc5ec0e service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:54.653180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fd665-f3d8-4989-9576-21381dc5ec0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:54.654823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-405fd665-f3d8-4989-9576-21381dc5ec0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:54.705245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:54.747826 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fd665-f3d8-4989-9576-21381dc5ec0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:54.749461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405fd665-f3d8-4989-9576-21381dc5ec0e service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:54.768112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 532/1053] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:24:57.134900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebe17-6d00-4e09-bf7b-43aa40e1653c None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:57.135409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebe17-6d00-4e09-bf7b-43aa40e1653c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:57.135517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebe17-6d00-4e09-bf7b-43aa40e1653c None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:57.137022 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 522/1054] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:24:57.142794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b867d-daf9-44a8-97c8-dd131c5be6d9 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:57.143247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b867d-daf9-44a8-97c8-dd131c5be6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:57.143330 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b867d-daf9-44a8-97c8-dd131c5be6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:57.209907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-ec6b867d-daf9-44a8-97c8-dd131c5be6d9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:24:57.211134 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ec6b867d-daf9-44a8-97c8-dd131c5be6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:57.309553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 533/1055] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:24:57.608396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadebe17-6d00-4e09-bf7b-43aa40e1653c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:24:57.610145 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-fadebe17-6d00-4e09-bf7b-43aa40e1653c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:57.649176 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:57.762400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadebe17-6d00-4e09-bf7b-43aa40e1653c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:24:57.881657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:24:57.886860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b96b7-fbfe-48ed-825c-e1896b1d2e28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:24:57.887462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b96b7-fbfe-48ed-825c-e1896b1d2e28 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:24:57.887734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b96b7-fbfe-48ed-825c-e1896b1d2e28 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:24:57.888001 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b96b7-fbfe-48ed-825c-e1896b1d2e28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:24:57.889664 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-fd8b96b7-fbfe-48ed-825c-e1896b1d2e28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:24:57.986669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b96b7-fbfe-48ed-825c-e1896b1d2e28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:24:57.988412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b96b7-fbfe-48ed-825c-e1896b1d2e28 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:24:58.007888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 523/1056] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:00.401758 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ab91-b433-4ef3-bfac-d32551cf1751 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:00.402286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ab91-b433-4ef3-bfac-d32551cf1751 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:00.402452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ab91-b433-4ef3-bfac-d32551cf1751 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:00.403666 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 534/1057] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:00.409703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f704b-801b-4f42-8793-7b5b1b07c5e0 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:00.410241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f704b-801b-4f42-8793-7b5b1b07c5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:00.410241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f704b-801b-4f42-8793-7b5b1b07c5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:00.488832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-f04f704b-801b-4f42-8793-7b5b1b07c5e0 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:00.490034 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f04f704b-801b-4f42-8793-7b5b1b07c5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:00.561787 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:00.604948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 524/1058] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:00.983668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652ab91-b433-4ef3-bfac-d32551cf1751 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:00.986191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f652ab91-b433-4ef3-bfac-d32551cf1751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:01.031017 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:01.184362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652ab91-b433-4ef3-bfac-d32551cf1751 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:01.289444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:01.306671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7bd86-81ac-40f2-87c5-0a27144dcb42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:01.307281 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7bd86-81ac-40f2-87c5-0a27144dcb42 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:01.307556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7bd86-81ac-40f2-87c5-0a27144dcb42 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:01.307823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7bd86-81ac-40f2-87c5-0a27144dcb42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:01.309418 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3ff7bd86-81ac-40f2-87c5-0a27144dcb42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:01.410252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7bd86-81ac-40f2-87c5-0a27144dcb42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:01.411866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7bd86-81ac-40f2-87c5-0a27144dcb42 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:01.430662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 535/1059] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:03.768093 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994715f-6c12-4fb3-8e28-ca285018a93d None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:03.768924 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994715f-6c12-4fb3-8e28-ca285018a93d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:03.768924 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994715f-6c12-4fb3-8e28-ca285018a93d None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:03.770143 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 525/1060] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:03.775489 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3ee80-470d-44af-bda0-4e2340e35db5 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:03.776050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3ee80-470d-44af-bda0-4e2340e35db5 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:03.776217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3ee80-470d-44af-bda0-4e2340e35db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:03.853207 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b2f3ee80-470d-44af-bda0-4e2340e35db5 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:03.854244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b2f3ee80-470d-44af-bda0-4e2340e35db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:03.917435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:03.957163 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 536/1061] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:04.253132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994715f-6c12-4fb3-8e28-ca285018a93d None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:04.255376 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d994715f-6c12-4fb3-8e28-ca285018a93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:04.420825 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994715f-6c12-4fb3-8e28-ca285018a93d None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:04.475994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:04.538149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e03f57-5de9-49b5-930d-1ec9e8decc13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:04.538761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e03f57-5de9-49b5-930d-1ec9e8decc13 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:04.539028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e03f57-5de9-49b5-930d-1ec9e8decc13 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:04.539302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e03f57-5de9-49b5-930d-1ec9e8decc13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:04.540887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-18e03f57-5de9-49b5-930d-1ec9e8decc13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:04.632499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e03f57-5de9-49b5-930d-1ec9e8decc13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:04.634109 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e03f57-5de9-49b5-930d-1ec9e8decc13 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:04.652355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 526/1062] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:06.867670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0073b97-5b27-4aef-a936-cf5d72b19070 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:06.868232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0073b97-5b27-4aef-a936-cf5d72b19070 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:06.868232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0073b97-5b27-4aef-a936-cf5d72b19070 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:06.869529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 537/1063] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:06.875231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e2ec3-e00c-4373-bc2d-171447069666 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:06.875644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e2ec3-e00c-4373-bc2d-171447069666 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:06.875775 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e2ec3-e00c-4373-bc2d-171447069666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:06.942147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-e30e2ec3-e00c-4373-bc2d-171447069666 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:06.943374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e30e2ec3-e00c-4373-bc2d-171447069666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:06.951547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:07.038656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 527/1064] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:07.325694 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0073b97-5b27-4aef-a936-cf5d72b19070 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:07.327836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c0073b97-5b27-4aef-a936-cf5d72b19070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:07.472571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0073b97-5b27-4aef-a936-cf5d72b19070 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:07.517539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:07.588046 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f178f72-abcc-47a8-81b0-fa58a3407e53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:07.588558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f178f72-abcc-47a8-81b0-fa58a3407e53 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:07.588830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f178f72-abcc-47a8-81b0-fa58a3407e53 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:07.589094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f178f72-abcc-47a8-81b0-fa58a3407e53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:07.590668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9f178f72-abcc-47a8-81b0-fa58a3407e53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:07.682926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f178f72-abcc-47a8-81b0-fa58a3407e53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:07.684520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f178f72-abcc-47a8-81b0-fa58a3407e53 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:07.703597 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 538/1065] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:08.514351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c0fd-225a-4385-b239-b11cdb1347a9 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:08.514944 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c0fd-225a-4385-b239-b11cdb1347a9 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:08.514944 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376c0fd-225a-4385-b239-b11cdb1347a9 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:08.516232 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 528/1066] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:08.521458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1d09b-2b09-465b-9cda-fa6b1c8eb639 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:08.521908 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1d09b-2b09-465b-9cda-fa6b1c8eb639 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:08.522012 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd1d09b-2b09-465b-9cda-fa6b1c8eb639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:08.587997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:08.591083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8cd1d09b-2b09-465b-9cda-fa6b1c8eb639 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:08.592807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8cd1d09b-2b09-465b-9cda-fa6b1c8eb639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:08.696777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 539/1067] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:09.046604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475feea-5269-42de-b1b3-b02ec426c7c2 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:09.046993 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475feea-5269-42de-b1b3-b02ec426c7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:09.047277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475feea-5269-42de-b1b3-b02ec426c7c2 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:09.048613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 529/1068] 10.222.0.20 () {54 vars in 962 bytes} [Wed Oct 19 02:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:09.053995 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b63ee-da6f-40fe-9d3c-3b2db169a2ba None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:09.054338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b63ee-da6f-40fe-9d3c-3b2db169a2ba None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:09.054619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b63ee-da6f-40fe-9d3c-3b2db169a2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:09.136447 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8c4b63ee-da6f-40fe-9d3c-3b2db169a2ba None None] MFA Rules not processed for user `286cfcbb8334421f87c0d426aa52a30a`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:09.137645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8c4b63ee-da6f-40fe-9d3c-3b2db169a2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:09.207669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:09.213236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 540/1069] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Oct 19 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:25:09.221924 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475feea-5269-42de-b1b3-b02ec426c7c2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:09.223507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7475feea-5269-42de-b1b3-b02ec426c7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:09.346528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475feea-5269-42de-b1b3-b02ec426c7c2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:09.388698 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:09.425664 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32bcc6-35ec-4818-a578-639e93883f41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286cfcbb8334421f87c0d426aa52a30a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:09.426309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32bcc6-35ec-4818-a578-639e93883f41 service glance] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:09.426621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32bcc6-35ec-4818-a578-639e93883f41 service glance] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:09.426905 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32bcc6-35ec-4818-a578-639e93883f41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:09.428484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5b32bcc6-35ec-4818-a578-639e93883f41 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:09.540238 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32bcc6-35ec-4818-a578-639e93883f41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:09.541667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32bcc6-35ec-4818-a578-639e93883f41 service glance] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:09.563346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 530/1070] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:10.042872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36602d-fa4c-496d-adca-bd4b3d848e2b None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:10.043513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36602d-fa4c-496d-adca-bd4b3d848e2b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:10.043513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36602d-fa4c-496d-adca-bd4b3d848e2b None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:10.044945 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 541/1071] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:10.050451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99c668-ab84-4a2d-94ef-91006b55b4b0 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:10.050748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99c668-ab84-4a2d-94ef-91006b55b4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:10.051024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99c668-ab84-4a2d-94ef-91006b55b4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:10.125168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-8b99c668-ab84-4a2d-94ef-91006b55b4b0 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:10.125847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8b99c668-ab84-4a2d-94ef-91006b55b4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:10.146085 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:10.226091 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 531/1072] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:25:10.516060 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36602d-fa4c-496d-adca-bd4b3d848e2b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:10.517583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4d36602d-fa4c-496d-adca-bd4b3d848e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:10.536582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b538bc-8cae-4a11-8bec-35bc9b6d7159 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:10.536866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b538bc-8cae-4a11-8bec-35bc9b6d7159 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:10.537166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b538bc-8cae-4a11-8bec-35bc9b6d7159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:10.607615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-95b538bc-8cae-4a11-8bec-35bc9b6d7159 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:10.609408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-95b538bc-8cae-4a11-8bec-35bc9b6d7159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:10.671956 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36602d-fa4c-496d-adca-bd4b3d848e2b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:10.697157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 532/1073] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:25:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:10.740609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:10.745897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8067f-24b5-41ca-ad87-cf75953280da None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:10.746552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8067f-24b5-41ca-ad87-cf75953280da None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:10.746894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8067f-24b5-41ca-ad87-cf75953280da None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:10.749253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 533/1074] 10.222.0.20 () {54 vars in 943 bytes} [Wed Oct 19 02:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:10.756816 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c6eff-40c7-42c4-aae6-2488b9621e27 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:10.757171 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c6eff-40c7-42c4-aae6-2488b9621e27 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:10.757519 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c6eff-40c7-42c4-aae6-2488b9621e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:10.775772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:10.789631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c8f44-5cf1-4028-9811-9a89668a7470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:10.790272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c8f44-5cf1-4028-9811-9a89668a7470 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:10.790551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c8f44-5cf1-4028-9811-9a89668a7470 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:10.790819 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c8f44-5cf1-4028-9811-9a89668a7470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:10.792424 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-247c8f44-5cf1-4028-9811-9a89668a7470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:10.840876 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-a35c6eff-40c7-42c4-aae6-2488b9621e27 None None] MFA Rules not processed for user `e476a32b458c4ea0bdc207d0ff4bfd47`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:10.842136 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a35c6eff-40c7-42c4-aae6-2488b9621e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:10.885641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c8f44-5cf1-4028-9811-9a89668a7470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:10.887276 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247c8f44-5cf1-4028-9811-9a89668a7470 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:10.906257 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 542/1075] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:10.950401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 534/1076] 10.222.0.20 () {58 vars in 1040 bytes} [Wed Oct 19 02:25:10 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:10.960068 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c8f44-5cf1-4028-9811-9a89668a7470 service nova] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:10.961475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-247c8f44-5cf1-4028-9811-9a89668a7470 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:11.047619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:11.051688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c8f44-5cf1-4028-9811-9a89668a7470 service nova] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:11.173442 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefcf40-42d4-4e5b-9060-b9eef5448f20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:11.174083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefcf40-42d4-4e5b-9060-b9eef5448f20 service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:11.174515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefcf40-42d4-4e5b-9060-b9eef5448f20 service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:11.174796 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefcf40-42d4-4e5b-9060-b9eef5448f20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:11.176510 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6eefcf40-42d4-4e5b-9060-b9eef5448f20 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:11.266368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefcf40-42d4-4e5b-9060-b9eef5448f20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:11.267965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefcf40-42d4-4e5b-9060-b9eef5448f20 service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:11.276655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 543/1077] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:25:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:25:13.354031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4277777-fad1-4fda-9e34-c193a6da69c6 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:13.355078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4277777-fad1-4fda-9e34-c193a6da69c6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:13.356578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4277777-fad1-4fda-9e34-c193a6da69c6 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:13.358756 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 535/1078] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:25:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:13.364312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d774e-51ba-483b-9ca9-5d4223d5f177 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:13.365089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d774e-51ba-483b-9ca9-5d4223d5f177 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:13.365559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d774e-51ba-483b-9ca9-5d4223d5f177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:13.405309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:13.441325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e41d774e-51ba-483b-9ca9-5d4223d5f177 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:13.442563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e41d774e-51ba-483b-9ca9-5d4223d5f177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:13.550454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 544/1079] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:13.841450 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4277777-fad1-4fda-9e34-c193a6da69c6 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:13.843121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c4277777-fad1-4fda-9e34-c193a6da69c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:13.927806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:14.011755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4277777-fad1-4fda-9e34-c193a6da69c6 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:14.141566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be5dab9-3084-4100-b678-b8c0f991a7dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:14.142320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5dab9-3084-4100-b678-b8c0f991a7dd service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:14.142595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5dab9-3084-4100-b678-b8c0f991a7dd service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:14.142867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be5dab9-3084-4100-b678-b8c0f991a7dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:14.144444 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8be5dab9-3084-4100-b678-b8c0f991a7dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:14.183662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:14.253637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5dab9-3084-4100-b678-b8c0f991a7dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:14.255467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be5dab9-3084-4100-b678-b8c0f991a7dd service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:14.275112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 536/1080] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:16.668850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229a6b-1f10-4b29-b9c0-8d066cdc5ab2 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:16.669459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229a6b-1f10-4b29-b9c0-8d066cdc5ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:16.669459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229a6b-1f10-4b29-b9c0-8d066cdc5ab2 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:16.670956 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 545/1081] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:16.676264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc83616-2527-417b-b0a4-31bf8e937b02 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:16.676767 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc83616-2527-417b-b0a4-31bf8e937b02 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:16.676861 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc83616-2527-417b-b0a4-31bf8e937b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:16.749344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7fc83616-2527-417b-b0a4-31bf8e937b02 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:16.750606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7fc83616-2527-417b-b0a4-31bf8e937b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:16.862187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 537/1082] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:17.146537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07229a6b-1f10-4b29-b9c0-8d066cdc5ab2 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:17.148043 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-07229a6b-1f10-4b29-b9c0-8d066cdc5ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:17.259241 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:17.307389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07229a6b-1f10-4b29-b9c0-8d066cdc5ab2 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:17.443320 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011b467-ef97-4c67-a6b7-4bbd5060af40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:17.443869 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011b467-ef97-4c67-a6b7-4bbd5060af40 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:17.444150 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011b467-ef97-4c67-a6b7-4bbd5060af40 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:17.444425 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011b467-ef97-4c67-a6b7-4bbd5060af40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:17.446073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4011b467-ef97-4c67-a6b7-4bbd5060af40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:17.545347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011b467-ef97-4c67-a6b7-4bbd5060af40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:17.547028 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4011b467-ef97-4c67-a6b7-4bbd5060af40 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:17.565630 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 546/1083] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:17.638658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:20.479127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8829957-f541-48f8-b017-984e4b7ee5a0 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:20.479683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8829957-f541-48f8-b017-984e4b7ee5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:20.479683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8829957-f541-48f8-b017-984e4b7ee5a0 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:20.481433 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 538/1084] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:20.486693 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81e0c9-4209-4068-b0c3-45818333c6a0 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:20.487434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81e0c9-4209-4068-b0c3-45818333c6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:20.487877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81e0c9-4209-4068-b0c3-45818333c6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:20.563215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8a81e0c9-4209-4068-b0c3-45818333c6a0 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:20.564436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8a81e0c9-4209-4068-b0c3-45818333c6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:20.669565 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 547/1085] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:20.958981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8829957-f541-48f8-b017-984e4b7ee5a0 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:20.960178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a8829957-f541-48f8-b017-984e4b7ee5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:20.971960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:21.130148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8829957-f541-48f8-b017-984e4b7ee5a0 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:21.220005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:21.253886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03f133-ec5a-4fc1-b4b2-d7ba2cd31143 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:21.254524 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f133-ec5a-4fc1-b4b2-d7ba2cd31143 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:21.254807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f133-ec5a-4fc1-b4b2-d7ba2cd31143 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:21.255080 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03f133-ec5a-4fc1-b4b2-d7ba2cd31143 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:21.257076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3b03f133-ec5a-4fc1-b4b2-d7ba2cd31143 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:21.357522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03f133-ec5a-4fc1-b4b2-d7ba2cd31143 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:21.359494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03f133-ec5a-4fc1-b4b2-d7ba2cd31143 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:21.382310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 539/1086] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:30.667368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=79 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:31.377505 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=58 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:35.480557 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943e166-44bf-49e3-b7d9-ae7cee7d3634 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:35.481110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943e166-44bf-49e3-b7d9-ae7cee7d3634 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:35.481209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943e166-44bf-49e3-b7d9-ae7cee7d3634 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:35.482962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 548/1087] 10.222.0.20 () {54 vars in 918 bytes} [Wed Oct 19 02:25:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:35.488654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b2758-a544-4164-a5db-e87a5fdde833 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:35.489169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b2758-a544-4164-a5db-e87a5fdde833 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:35.489169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b2758-a544-4164-a5db-e87a5fdde833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:35.570966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-625b2758-a544-4164-a5db-e87a5fdde833 None None] MFA Rules not processed for user `9243ebe24ae84157b74696c29bda3279`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:35.572504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-625b2758-a544-4164-a5db-e87a5fdde833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:35.680286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 540/1088] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Oct 19 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:35.695328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae6716-38cd-4b50-b416-2425939aa4ad None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:35.695613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae6716-38cd-4b50-b416-2425939aa4ad None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:35.695885 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae6716-38cd-4b50-b416-2425939aa4ad None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:35.697211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 549/1089] 10.222.0.20 () {54 vars in 918 bytes} [Wed Oct 19 02:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:35.701928 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdd3a7-7f7f-4102-9709-8834da675b7c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:35.702245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdd3a7-7f7f-4102-9709-8834da675b7c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:35.702521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdd3a7-7f7f-4102-9709-8834da675b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:35.780645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:35.785067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-55bdd3a7-7f7f-4102-9709-8834da675b7c None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:35.786682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-55bdd3a7-7f7f-4102-9709-8834da675b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:35.904777 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 541/1090] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Oct 19 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:38.481177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9f1ac-90a6-4416-b887-84f926774270 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:38.481689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9f1ac-90a6-4416-b887-84f926774270 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:38.481689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9f1ac-90a6-4416-b887-84f926774270 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:38.483242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 550/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:38.489018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9dfe-cfc7-45d1-aa44-fb142207f932 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:38.489436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9dfe-cfc7-45d1-aa44-fb142207f932 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:38.489563 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a9dfe-cfc7-45d1-aa44-fb142207f932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:38.554178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-828a9dfe-cfc7-45d1-aa44-fb142207f932 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:38.555379 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-828a9dfe-cfc7-45d1-aa44-fb142207f932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:38.610003 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:38.650118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 542/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:38.943943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9f1ac-90a6-4416-b887-84f926774270 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:38.945451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c1c9f1ac-90a6-4416-b887-84f926774270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:39.088028 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9f1ac-90a6-4416-b887-84f926774270 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:39.160569 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:39.200029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acec4b9-1310-4393-b4c2-9d2238f97746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:39.200567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec4b9-1310-4393-b4c2-9d2238f97746 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:39.200844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec4b9-1310-4393-b4c2-9d2238f97746 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:39.201163 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acec4b9-1310-4393-b4c2-9d2238f97746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:39.202770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6acec4b9-1310-4393-b4c2-9d2238f97746 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:39.294311 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acec4b9-1310-4393-b4c2-9d2238f97746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:39.295930 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acec4b9-1310-4393-b4c2-9d2238f97746 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:39.314542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 551/1093] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:40.455695 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476555b-1842-41ca-9394-7b8621ece12d None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:40.456221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476555b-1842-41ca-9394-7b8621ece12d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:40.456221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476555b-1842-41ca-9394-7b8621ece12d None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:40.457485 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 543/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:40.463641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db170c-6a6b-4d72-a3f3-d1f1d5a93d67 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:40.464093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db170c-6a6b-4d72-a3f3-d1f1d5a93d67 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:40.464200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db170c-6a6b-4d72-a3f3-d1f1d5a93d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:40.539114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c1db170c-6a6b-4d72-a3f3-d1f1d5a93d67 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:40.540309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c1db170c-6a6b-4d72-a3f3-d1f1d5a93d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:40.566725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:40.637194 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 552/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:40.800498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476555b-1842-41ca-9394-7b8621ece12d None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:40.802007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3476555b-1842-41ca-9394-7b8621ece12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:40.944044 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476555b-1842-41ca-9394-7b8621ece12d None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:40.983998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:41.054348 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1985db-8359-421c-8df3-9ddbf525167f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:41.054880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1985db-8359-421c-8df3-9ddbf525167f service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:41.055152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1985db-8359-421c-8df3-9ddbf525167f service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:41.055418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1985db-8359-421c-8df3-9ddbf525167f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:41.056981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3d1985db-8359-421c-8df3-9ddbf525167f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:41.146596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1985db-8359-421c-8df3-9ddbf525167f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:41.148176 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1985db-8359-421c-8df3-9ddbf525167f service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:41.165598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 544/1096] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:42.286142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113c31b-8ed2-48b1-afe5-077650263b32 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:42.286688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113c31b-8ed2-48b1-afe5-077650263b32 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:42.286688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113c31b-8ed2-48b1-afe5-077650263b32 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:42.288037 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 553/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:42.294353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a46271-8a03-4696-bbcb-b5936b97a058 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:42.294774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a46271-8a03-4696-bbcb-b5936b97a058 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:42.294903 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a46271-8a03-4696-bbcb-b5936b97a058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:42.348137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:42.369823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-71a46271-8a03-4696-bbcb-b5936b97a058 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:42.371060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-71a46271-8a03-4696-bbcb-b5936b97a058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:42.465506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 545/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:42.621963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113c31b-8ed2-48b1-afe5-077650263b32 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:42.621963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1113c31b-8ed2-48b1-afe5-077650263b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:42.768270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113c31b-8ed2-48b1-afe5-077650263b32 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:42.780002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:42.881031 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2f127-a377-418c-9aa0-da5b9316d67e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:42.881543 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f127-a377-418c-9aa0-da5b9316d67e service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:42.881820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f127-a377-418c-9aa0-da5b9316d67e service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:42.882113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f127-a377-418c-9aa0-da5b9316d67e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:42.883678 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-56f2f127-a377-418c-9aa0-da5b9316d67e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:42.969165 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f127-a377-418c-9aa0-da5b9316d67e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:42.970766 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f127-a377-418c-9aa0-da5b9316d67e service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:42.988150 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 554/1099] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:44.097766 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60407ddc-0a75-47b0-ab92-45cb03d48f7e None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:44.105153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60407ddc-0a75-47b0-ab92-45cb03d48f7e None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:44.105153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60407ddc-0a75-47b0-ab92-45cb03d48f7e None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:44.105153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 546/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:44.106222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7586ef-e7fa-4c9e-ae24-7543fc379a87 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:44.106645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7586ef-e7fa-4c9e-ae24-7543fc379a87 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:44.106645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7586ef-e7fa-4c9e-ae24-7543fc379a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:44.126307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:44.181794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-de7586ef-e7fa-4c9e-ae24-7543fc379a87 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:44.183056 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-de7586ef-e7fa-4c9e-ae24-7543fc379a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:44.279981 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 555/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:44.432970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60407ddc-0a75-47b0-ab92-45cb03d48f7e None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:44.434501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-60407ddc-0a75-47b0-ab92-45cb03d48f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:44.544492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:44.580521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60407ddc-0a75-47b0-ab92-45cb03d48f7e None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:44.690657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46b0e3-bf77-457b-ba24-59b19049ffd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:44.691181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46b0e3-bf77-457b-ba24-59b19049ffd7 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:44.691458 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46b0e3-bf77-457b-ba24-59b19049ffd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:44.691728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46b0e3-bf77-457b-ba24-59b19049ffd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:44.693298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-db46b0e3-bf77-457b-ba24-59b19049ffd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:44.769004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:44.781924 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46b0e3-bf77-457b-ba24-59b19049ffd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:44.786556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46b0e3-bf77-457b-ba24-59b19049ffd7 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:44.801452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 547/1102] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:45.917703 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe4f60-d6af-4d52-ae91-665bed2998b4 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:45.918288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe4f60-d6af-4d52-ae91-665bed2998b4 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:45.918288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe4f60-d6af-4d52-ae91-665bed2998b4 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:45.919684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 556/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:45.926148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f3c55-e181-49e4-828b-21f879814917 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:45.926582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f3c55-e181-49e4-828b-21f879814917 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:45.926687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f3c55-e181-49e4-828b-21f879814917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:46.001167 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ca6f3c55-e181-49e4-828b-21f879814917 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:46.002389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ca6f3c55-e181-49e4-828b-21f879814917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:46.095412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 548/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:46.248053 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe4f60-d6af-4d52-ae91-665bed2998b4 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:46.249599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-dcbe4f60-d6af-4d52-ae91-665bed2998b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:46.332271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:46.393704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe4f60-d6af-4d52-ae91-665bed2998b4 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:46.505779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56f0d0-01bb-4edc-b967-ed710f9d2655 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:46.506345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56f0d0-01bb-4edc-b967-ed710f9d2655 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:46.506623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56f0d0-01bb-4edc-b967-ed710f9d2655 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:46.506893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56f0d0-01bb-4edc-b967-ed710f9d2655 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:46.508464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1f56f0d0-01bb-4edc-b967-ed710f9d2655 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:46.556461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:46.596642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56f0d0-01bb-4edc-b967-ed710f9d2655 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:46.598255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56f0d0-01bb-4edc-b967-ed710f9d2655 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:46.616392 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 557/1105] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:47.774460 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaa7f2-45e3-4a15-b420-37d6a0fa8009 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:47.774899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaa7f2-45e3-4a15-b420-37d6a0fa8009 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:47.774992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdaa7f2-45e3-4a15-b420-37d6a0fa8009 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:47.776350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 549/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:47.783183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a5c56-6dea-4601-8ddd-a22ff1903c3b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:47.783690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a5c56-6dea-4601-8ddd-a22ff1903c3b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:47.783796 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a5c56-6dea-4601-8ddd-a22ff1903c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:47.858409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-4e2a5c56-6dea-4601-8ddd-a22ff1903c3b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:47.859650 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4e2a5c56-6dea-4601-8ddd-a22ff1903c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:47.960761 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 558/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:48.112104 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdaa7f2-45e3-4a15-b420-37d6a0fa8009 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:48.113655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6cdaa7f2-45e3-4a15-b420-37d6a0fa8009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:48.161873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:48.257879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdaa7f2-45e3-4a15-b420-37d6a0fa8009 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:48.368268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0650924f-1b02-486f-a5ff-82d1f1978f69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:48.368783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650924f-1b02-486f-a5ff-82d1f1978f69 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:48.369056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650924f-1b02-486f-a5ff-82d1f1978f69 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:48.369323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0650924f-1b02-486f-a5ff-82d1f1978f69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:48.370942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0650924f-1b02-486f-a5ff-82d1f1978f69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:48.377772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:48.460082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650924f-1b02-486f-a5ff-82d1f1978f69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:48.461647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0650924f-1b02-486f-a5ff-82d1f1978f69 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:48.479445 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 550/1108] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:49.574738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0d540-5422-44da-8e56-bbc183a94671 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:49.575277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0d540-5422-44da-8e56-bbc183a94671 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:49.575277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0d540-5422-44da-8e56-bbc183a94671 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:49.576616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 559/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:49.583166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3673e0-3645-4b2a-adba-2a45abb62c90 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:49.583628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3673e0-3645-4b2a-adba-2a45abb62c90 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:49.583739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3673e0-3645-4b2a-adba-2a45abb62c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:49.657910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ee3673e0-3645-4b2a-adba-2a45abb62c90 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:49.659136 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ee3673e0-3645-4b2a-adba-2a45abb62c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:49.724206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:49.752437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 551/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:49.907022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0d540-5422-44da-8e56-bbc183a94671 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:49.908463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0ae0d540-5422-44da-8e56-bbc183a94671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:49.926167 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:50.051256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0d540-5422-44da-8e56-bbc183a94671 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:50.138854 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:50.169409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964acf8c-c502-45cd-8a58-63f70c4bb248 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:50.169409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964acf8c-c502-45cd-8a58-63f70c4bb248 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:50.169409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964acf8c-c502-45cd-8a58-63f70c4bb248 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:50.169409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964acf8c-c502-45cd-8a58-63f70c4bb248 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:50.169873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-964acf8c-c502-45cd-8a58-63f70c4bb248 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:50.254655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964acf8c-c502-45cd-8a58-63f70c4bb248 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:50.256228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964acf8c-c502-45cd-8a58-63f70c4bb248 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:50.274087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 560/1111] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:51.417240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af562b-ae85-49f9-a801-8460ffa07747 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:51.417757 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af562b-ae85-49f9-a801-8460ffa07747 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:51.417757 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60af562b-ae85-49f9-a801-8460ffa07747 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:51.419180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 552/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:51.425411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5965bb-b4b1-4703-bdf2-5e6d842255f2 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:51.425857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5965bb-b4b1-4703-bdf2-5e6d842255f2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:51.425954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5965bb-b4b1-4703-bdf2-5e6d842255f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:51.501658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-fa5965bb-b4b1-4703-bdf2-5e6d842255f2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:51.502866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-fa5965bb-b4b1-4703-bdf2-5e6d842255f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:51.542467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:51.595286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 561/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:51.745762 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af562b-ae85-49f9-a801-8460ffa07747 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:51.747311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-60af562b-ae85-49f9-a801-8460ffa07747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:51.889629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60af562b-ae85-49f9-a801-8460ffa07747 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:51.949146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:52.008581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05fbc7-2ec6-4052-85b3-d11cf1a78d82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:52.009105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05fbc7-2ec6-4052-85b3-d11cf1a78d82 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:52.009379 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05fbc7-2ec6-4052-85b3-d11cf1a78d82 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:52.009652 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05fbc7-2ec6-4052-85b3-d11cf1a78d82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:52.011237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1b05fbc7-2ec6-4052-85b3-d11cf1a78d82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:52.097720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05fbc7-2ec6-4052-85b3-d11cf1a78d82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:52.099400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05fbc7-2ec6-4052-85b3-d11cf1a78d82 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:52.118508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 553/1114] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:25:53.226018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10009e4f-7aef-4229-ab9f-a27b50601370 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:53.226590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10009e4f-7aef-4229-ab9f-a27b50601370 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:53.226590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10009e4f-7aef-4229-ab9f-a27b50601370 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:53.227916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 562/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:53.234263 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec72359-9b3c-441b-95e0-2393eb82bdfc None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:53.234719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec72359-9b3c-441b-95e0-2393eb82bdfc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:53.234842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec72359-9b3c-441b-95e0-2393eb82bdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:53.308243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7ec72359-9b3c-441b-95e0-2393eb82bdfc None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:53.309434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7ec72359-9b3c-441b-95e0-2393eb82bdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:53.311423 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:53.404758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 554/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:53.550397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10009e4f-7aef-4229-ab9f-a27b50601370 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:53.551961 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-10009e4f-7aef-4229-ab9f-a27b50601370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:53.695064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10009e4f-7aef-4229-ab9f-a27b50601370 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:53.720921 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:53.803965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a84f587-402e-4aa2-a1df-6c03edfc505e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:53.804556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84f587-402e-4aa2-a1df-6c03edfc505e service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:53.804826 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84f587-402e-4aa2-a1df-6c03edfc505e service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:53.805089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84f587-402e-4aa2-a1df-6c03edfc505e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:53.806736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0a84f587-402e-4aa2-a1df-6c03edfc505e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:53.896489 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84f587-402e-4aa2-a1df-6c03edfc505e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:53.898212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a84f587-402e-4aa2-a1df-6c03edfc505e service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:53.923032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 563/1117] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:55.981636 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53547834-2ae5-4d0f-9e56-7a147b788c9f None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:55.982344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53547834-2ae5-4d0f-9e56-7a147b788c9f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:55.982344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53547834-2ae5-4d0f-9e56-7a147b788c9f None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:55.983812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 555/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:55.989106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db3a98-409d-46c5-9b6f-854092467e39 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:55.989550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db3a98-409d-46c5-9b6f-854092467e39 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:55.989678 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db3a98-409d-46c5-9b6f-854092467e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:56.029947 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:56.054493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-d5db3a98-409d-46c5-9b6f-854092467e39 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:56.055670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d5db3a98-409d-46c5-9b6f-854092467e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:56.151630 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 564/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:56.433656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53547834-2ae5-4d0f-9e56-7a147b788c9f None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:56.434805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-53547834-2ae5-4d0f-9e56-7a147b788c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:56.536782 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53547834-2ae5-4d0f-9e56-7a147b788c9f None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:56.567562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:56.608643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ffc5b-5115-4c17-b80f-888c87919391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:56.609239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ffc5b-5115-4c17-b80f-888c87919391 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:56.609506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ffc5b-5115-4c17-b80f-888c87919391 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:56.609765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ffc5b-5115-4c17-b80f-888c87919391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:56.611349 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5f2ffc5b-5115-4c17-b80f-888c87919391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:56.717833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ffc5b-5115-4c17-b80f-888c87919391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:56.719277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ffc5b-5115-4c17-b80f-888c87919391 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:56.737360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 556/1120] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:25:59.051603 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2d109-7cf0-4d52-8970-19dc7069cdcf None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:59.052171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2d109-7cf0-4d52-8970-19dc7069cdcf None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:59.052275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2d109-7cf0-4d52-8970-19dc7069cdcf None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:59.053610 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 565/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:25:59.058898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9435f-61f1-45b3-8706-64c4bc079018 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:59.059296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9435f-61f1-45b3-8706-64c4bc079018 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:59.059445 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9435f-61f1-45b3-8706-64c4bc079018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:59.126804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-e3f9435f-61f1-45b3-8706-64c4bc079018 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:25:59.127988 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e3f9435f-61f1-45b3-8706-64c4bc079018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:59.140296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:59.221340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 557/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:25:59.512605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2d109-7cf0-4d52-8970-19dc7069cdcf None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:25:59.514227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4bd2d109-7cf0-4d52-8970-19dc7069cdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:59.614903 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2d109-7cf0-4d52-8970-19dc7069cdcf None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:25:59.627400 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:25:59.687444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5c4e0-1c35-4c36-a46e-6b2f8ed334d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:25:59.688065 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5c4e0-1c35-4c36-a46e-6b2f8ed334d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:25:59.688340 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5c4e0-1c35-4c36-a46e-6b2f8ed334d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:25:59.688612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5c4e0-1c35-4c36-a46e-6b2f8ed334d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:25:59.690214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-84a5c4e0-1c35-4c36-a46e-6b2f8ed334d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:25:59.797122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5c4e0-1c35-4c36-a46e-6b2f8ed334d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:25:59.798508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5c4e0-1c35-4c36-a46e-6b2f8ed334d3 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:25:59.815931 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 566/1123] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:25:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:02.353480 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65833018-f29d-4867-ba09-3582f83197b5 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:02.354017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65833018-f29d-4867-ba09-3582f83197b5 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:02.354017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65833018-f29d-4867-ba09-3582f83197b5 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:02.355341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 558/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:02.361332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a998a-24f3-4717-90f6-226d9d7a3102 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:02.361757 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a998a-24f3-4717-90f6-226d9d7a3102 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:02.361867 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8a998a-24f3-4717-90f6-226d9d7a3102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:02.409443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:02.427765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-4a8a998a-24f3-4717-90f6-226d9d7a3102 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:02.428957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4a8a998a-24f3-4717-90f6-226d9d7a3102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:02.522148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 567/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:02.809081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65833018-f29d-4867-ba09-3582f83197b5 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:02.810655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-65833018-f29d-4867-ba09-3582f83197b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:02.911029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65833018-f29d-4867-ba09-3582f83197b5 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:02.950725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:02.982562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7c5f0-a0e4-4fc7-9b0a-0541551b990e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:02.983418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7c5f0-a0e4-4fc7-9b0a-0541551b990e service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:02.983693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7c5f0-a0e4-4fc7-9b0a-0541551b990e service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:02.983961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7c5f0-a0e4-4fc7-9b0a-0541551b990e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:02.986076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-59a7c5f0-a0e4-4fc7-9b0a-0541551b990e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:03.093455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7c5f0-a0e4-4fc7-9b0a-0541551b990e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:03.095344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7c5f0-a0e4-4fc7-9b0a-0541551b990e service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:03.114629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 559/1126] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:05.378196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a84a76-5c5c-4b14-9dcb-d6ba93d05bf6 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:05.378735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a84a76-5c5c-4b14-9dcb-d6ba93d05bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:05.378735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a84a76-5c5c-4b14-9dcb-d6ba93d05bf6 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:05.380032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 568/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:05.385243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579de4ed-f0c7-4c34-8b49-3a97d1065527 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:05.385663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579de4ed-f0c7-4c34-8b49-3a97d1065527 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:05.385819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579de4ed-f0c7-4c34-8b49-3a97d1065527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:05.449544 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-579de4ed-f0c7-4c34-8b49-3a97d1065527 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:05.450754 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-579de4ed-f0c7-4c34-8b49-3a97d1065527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:05.471042 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:05.545443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 560/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:05.833362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a84a76-5c5c-4b14-9dcb-d6ba93d05bf6 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:05.834911 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-93a84a76-5c5c-4b14-9dcb-d6ba93d05bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:05.936844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a84a76-5c5c-4b14-9dcb-d6ba93d05bf6 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:05.957322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:06.014858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e1900-96d2-4ba4-b109-1aef5fb3fba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:06.015463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e1900-96d2-4ba4-b109-1aef5fb3fba7 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:06.015768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e1900-96d2-4ba4-b109-1aef5fb3fba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:06.016036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e1900-96d2-4ba4-b109-1aef5fb3fba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:06.017629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-181e1900-96d2-4ba4-b109-1aef5fb3fba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:06.127165 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e1900-96d2-4ba4-b109-1aef5fb3fba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:06.128613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e1900-96d2-4ba4-b109-1aef5fb3fba7 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:06.146032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 569/1129] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:08.346995 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e94f0-afe3-45ed-a133-d71db204fe92 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:08.347538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e94f0-afe3-45ed-a133-d71db204fe92 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:08.347538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e94f0-afe3-45ed-a133-d71db204fe92 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:08.348855 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 561/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:08.354353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191f3f3-f2ae-4755-91f6-8519ff5f3d40 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:08.354775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191f3f3-f2ae-4755-91f6-8519ff5f3d40 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:08.354883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191f3f3-f2ae-4755-91f6-8519ff5f3d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:08.417634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:08.420427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8191f3f3-f2ae-4755-91f6-8519ff5f3d40 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:08.421616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8191f3f3-f2ae-4755-91f6-8519ff5f3d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:08.515584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 570/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:08.798567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e94f0-afe3-45ed-a133-d71db204fe92 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:08.800070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-bb5e94f0-afe3-45ed-a133-d71db204fe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:08.902578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e94f0-afe3-45ed-a133-d71db204fe92 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:08.950186 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:08.973736 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecf230-1d2c-4db0-afc3-db9bfd3ff143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:08.974357 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecf230-1d2c-4db0-afc3-db9bfd3ff143 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:08.974632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecf230-1d2c-4db0-afc3-db9bfd3ff143 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:08.974896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecf230-1d2c-4db0-afc3-db9bfd3ff143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:08.976453 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-32ecf230-1d2c-4db0-afc3-db9bfd3ff143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:09.083010 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecf230-1d2c-4db0-afc3-db9bfd3ff143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:09.084361 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecf230-1d2c-4db0-afc3-db9bfd3ff143 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:09.102466 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 562/1132] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:11.279138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03812c80-26c4-41d2-8a9a-707316af49d9 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:11.279691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03812c80-26c4-41d2-8a9a-707316af49d9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:11.279691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03812c80-26c4-41d2-8a9a-707316af49d9 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:11.281000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 571/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:11.286817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6fb3-e424-4161-94e5-a5664c42b292 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:11.287240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6fb3-e424-4161-94e5-a5664c42b292 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:11.287368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6fb3-e424-4161-94e5-a5664c42b292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:11.351343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-bd1f6fb3-e424-4161-94e5-a5664c42b292 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:11.352541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-bd1f6fb3-e424-4161-94e5-a5664c42b292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:11.384208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:11.451339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 563/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:26:11.730461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03812c80-26c4-41d2-8a9a-707316af49d9 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:11.732021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-03812c80-26c4-41d2-8a9a-707316af49d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:11.833212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03812c80-26c4-41d2-8a9a-707316af49d9 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:11.861197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:11.906337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcca8cd-0bfe-4cc9-894c-69ca323daf09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:11.906933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcca8cd-0bfe-4cc9-894c-69ca323daf09 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:11.907212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcca8cd-0bfe-4cc9-894c-69ca323daf09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:11.907479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcca8cd-0bfe-4cc9-894c-69ca323daf09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:11.909055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5fcca8cd-0bfe-4cc9-894c-69ca323daf09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:12.018261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcca8cd-0bfe-4cc9-894c-69ca323daf09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:12.020146 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcca8cd-0bfe-4cc9-894c-69ca323daf09 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:12.039870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 572/1135] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:14.238015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f588cb-9b91-4cfb-b8b0-c0c3425b62dc None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:14.238586 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f588cb-9b91-4cfb-b8b0-c0c3425b62dc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:14.238586 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f588cb-9b91-4cfb-b8b0-c0c3425b62dc None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:14.240157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 564/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:14.245397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8a0b5-f8f9-4ae6-9064-08f97ec1e3ac None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:14.245812 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8a0b5-f8f9-4ae6-9064-08f97ec1e3ac None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:14.245971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8a0b5-f8f9-4ae6-9064-08f97ec1e3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:14.310948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-1ca8a0b5-f8f9-4ae6-9064-08f97ec1e3ac None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:14.312147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1ca8a0b5-f8f9-4ae6-9064-08f97ec1e3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:14.320854 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:14.408269 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 573/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:14.694913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f588cb-9b91-4cfb-b8b0-c0c3425b62dc None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:14.696431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-31f588cb-9b91-4cfb-b8b0-c0c3425b62dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:14.796355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f588cb-9b91-4cfb-b8b0-c0c3425b62dc None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:14.849589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:14.868568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77605ba-eee1-4a06-89c1-07f026f967ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:14.869166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77605ba-eee1-4a06-89c1-07f026f967ba service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:14.869439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77605ba-eee1-4a06-89c1-07f026f967ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:14.869704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77605ba-eee1-4a06-89c1-07f026f967ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:14.871618 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c77605ba-eee1-4a06-89c1-07f026f967ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:14.978616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77605ba-eee1-4a06-89c1-07f026f967ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:14.979955 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77605ba-eee1-4a06-89c1-07f026f967ba service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:14.998566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 565/1138] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:17.193699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3aaab-20c1-4c56-a573-cdf70c55c153 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:17.194236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3aaab-20c1-4c56-a573-cdf70c55c153 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:17.194236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3aaab-20c1-4c56-a573-cdf70c55c153 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:17.195595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 574/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:17.200844 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6d980-b194-46e0-9758-1416eedb3489 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:17.201277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6d980-b194-46e0-9758-1416eedb3489 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:17.201390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6d980-b194-46e0-9758-1416eedb3489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:17.265374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-0aa6d980-b194-46e0-9758-1416eedb3489 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:17.267101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0aa6d980-b194-46e0-9758-1416eedb3489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:17.303980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:17.360906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 566/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:17.647283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3aaab-20c1-4c56-a573-cdf70c55c153 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:17.648855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-47b3aaab-20c1-4c56-a573-cdf70c55c153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:17.750270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3aaab-20c1-4c56-a573-cdf70c55c153 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:17.787293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:17.821387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e350a7d-259b-48d2-ae6a-de32eed39ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:17.822035 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e350a7d-259b-48d2-ae6a-de32eed39ada service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:17.822542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e350a7d-259b-48d2-ae6a-de32eed39ada service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:17.822965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e350a7d-259b-48d2-ae6a-de32eed39ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:17.824845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7e350a7d-259b-48d2-ae6a-de32eed39ada service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:17.931909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e350a7d-259b-48d2-ae6a-de32eed39ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:17.933251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e350a7d-259b-48d2-ae6a-de32eed39ada service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:17.951079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 575/1141] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:20.104851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16cb0-476b-40e2-af5f-22cd9624f4d7 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:20.105287 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16cb0-476b-40e2-af5f-22cd9624f4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:20.105381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc16cb0-476b-40e2-af5f-22cd9624f4d7 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:20.107120 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 567/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:20.112427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0f866-ab6e-4b2e-b3a5-b8d3c82b754e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:20.112851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0f866-ab6e-4b2e-b3a5-b8d3c82b754e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:20.112994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0f866-ab6e-4b2e-b3a5-b8d3c82b754e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:20.178063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-4ea0f866-ab6e-4b2e-b3a5-b8d3c82b754e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:20.179309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4ea0f866-ab6e-4b2e-b3a5-b8d3c82b754e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:20.197106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:20.274176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 576/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:20.555225 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc16cb0-476b-40e2-af5f-22cd9624f4d7 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:20.556740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-bdc16cb0-476b-40e2-af5f-22cd9624f4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:20.658178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc16cb0-476b-40e2-af5f-22cd9624f4d7 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:20.723974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:20.730465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcef556-54d0-47df-bc8d-058aa9d35f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:20.731070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcef556-54d0-47df-bc8d-058aa9d35f45 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:20.735174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcef556-54d0-47df-bc8d-058aa9d35f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:20.735174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcef556-54d0-47df-bc8d-058aa9d35f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:20.735174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-edcef556-54d0-47df-bc8d-058aa9d35f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:20.842449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcef556-54d0-47df-bc8d-058aa9d35f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:20.843795 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcef556-54d0-47df-bc8d-058aa9d35f45 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:20.862617 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 568/1144] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:23.008736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f63cf8-818d-430e-aeb3-fdda3658d92d None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:23.009280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f63cf8-818d-430e-aeb3-fdda3658d92d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:23.009280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f63cf8-818d-430e-aeb3-fdda3658d92d None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:23.010674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 577/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:26:23.016230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dae12-a17e-4b6c-a8ba-50e78027286e None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:23.016644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dae12-a17e-4b6c-a8ba-50e78027286e None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:23.016891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dae12-a17e-4b6c-a8ba-50e78027286e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:23.082289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-021dae12-a17e-4b6c-a8ba-50e78027286e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:23.083464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-021dae12-a17e-4b6c-a8ba-50e78027286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:23.133122 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:23.180025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 569/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:23.462230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f63cf8-818d-430e-aeb3-fdda3658d92d None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:23.463790 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-21f63cf8-818d-430e-aeb3-fdda3658d92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:23.564898 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f63cf8-818d-430e-aeb3-fdda3658d92d None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:23.611795 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:23.636562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb453606-867f-4f8f-89e3-c515c85b3785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:23.637096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb453606-867f-4f8f-89e3-c515c85b3785 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:23.637378 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb453606-867f-4f8f-89e3-c515c85b3785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:23.637648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb453606-867f-4f8f-89e3-c515c85b3785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:23.639290 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-fb453606-867f-4f8f-89e3-c515c85b3785 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:23.745477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb453606-867f-4f8f-89e3-c515c85b3785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:23.746951 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb453606-867f-4f8f-89e3-c515c85b3785 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:23.764345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 578/1147] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:33.178800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=22 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:33.762180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=66 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:43.899384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba453fa-f5fe-41b5-aefb-1c5b9242e64b None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:43.900011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba453fa-f5fe-41b5-aefb-1c5b9242e64b None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:43.900011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba453fa-f5fe-41b5-aefb-1c5b9242e64b None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:43.901451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 570/1148] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:43.907054 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f55e4-740c-44eb-9f17-815e08fccca3 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:43.910838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f55e4-740c-44eb-9f17-815e08fccca3 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:43.910838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f55e4-740c-44eb-9f17-815e08fccca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:43.972211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-347f55e4-740c-44eb-9f17-815e08fccca3 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:43.973453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-347f55e4-740c-44eb-9f17-815e08fccca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:44.067515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 579/1149] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:44.371578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4523ac-bcdf-4e18-b687-6c95f6091b76 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:44.371886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4523ac-bcdf-4e18-b687-6c95f6091b76 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:44.372181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4523ac-bcdf-4e18-b687-6c95f6091b76 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:44.373539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 571/1150] 10.222.0.20 () {54 vars in 962 bytes} [Wed Oct 19 02:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:44.378702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaed60-adcb-4f05-9833-cc5db73af9ea None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:44.378987 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaed60-adcb-4f05-9833-cc5db73af9ea None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:44.379256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaed60-adcb-4f05-9833-cc5db73af9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:44.455427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-14eaed60-adcb-4f05-9833-cc5db73af9ea None None] MFA Rules not processed for user `286cfcbb8334421f87c0d426aa52a30a`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:44.456679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-14eaed60-adcb-4f05-9833-cc5db73af9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:44.477366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:44.532476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 580/1151] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Oct 19 02:26:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:26:44.540579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4523ac-bcdf-4e18-b687-6c95f6091b76 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:44.542089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ea4523ac-bcdf-4e18-b687-6c95f6091b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:44.643046 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4523ac-bcdf-4e18-b687-6c95f6091b76 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:44.715148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08090c-a2c7-488a-9843-eece8266ed78 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286cfcbb8334421f87c0d426aa52a30a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:44.715778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08090c-a2c7-488a-9843-eece8266ed78 service glance] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:44.716059 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08090c-a2c7-488a-9843-eece8266ed78 service glance] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:44.716334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08090c-a2c7-488a-9843-eece8266ed78 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:44.717923 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2c08090c-a2c7-488a-9843-eece8266ed78 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:44.771085 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:44.824844 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08090c-a2c7-488a-9843-eece8266ed78 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:44.826299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08090c-a2c7-488a-9843-eece8266ed78 service glance] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:44.844709 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 572/1152] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:47.055750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179cce-90ff-456d-9521-3c615904792a None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:47.056314 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179cce-90ff-456d-9521-3c615904792a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:47.056314 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36179cce-90ff-456d-9521-3c615904792a None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:47.057671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 581/1153] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:47.062877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0401b1-5232-4292-af88-03b4a11fce7e None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:47.070587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0401b1-5232-4292-af88-03b4a11fce7e None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:47.070587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0401b1-5232-4292-af88-03b4a11fce7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:47.128965 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-1e0401b1-5232-4292-af88-03b4a11fce7e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:47.130210 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1e0401b1-5232-4292-af88-03b4a11fce7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:47.222629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 573/1154] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:47.499122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36179cce-90ff-456d-9521-3c615904792a None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:47.500703 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-36179cce-90ff-456d-9521-3c615904792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:47.624111 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36179cce-90ff-456d-9521-3c615904792a None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:47.670649 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:47.717948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f57b6-b944-4a80-8a2b-65bcf89fd0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:47.718572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f57b6-b944-4a80-8a2b-65bcf89fd0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:47.718844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f57b6-b944-4a80-8a2b-65bcf89fd0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:47.719161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f57b6-b944-4a80-8a2b-65bcf89fd0f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:47.720298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f57b6-b944-4a80-8a2b-65bcf89fd0f8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:47.721303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: WARNING py.warnings [None req-c51f57b6-b944-4a80-8a2b-65bcf89fd0f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:26:47.721303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: warnings.warn(msg) Oct 19 02:26:47.721303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: Oct 19 02:26:47.722472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f57b6-b944-4a80-8a2b-65bcf89fd0f8 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:47.741218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 582/1155] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Oct 19 02:26:47 2022] GET /identity/v3/projects => generated 2553 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:26:49.912917 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d7ae6-592a-4a5b-8112-d13625a08d0c None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:49.913479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d7ae6-592a-4a5b-8112-d13625a08d0c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:49.913479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d7ae6-592a-4a5b-8112-d13625a08d0c None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:49.914850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 574/1156] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:49.920040 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9869c8-a679-4630-ad50-ab7aa29fe142 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:49.920478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9869c8-a679-4630-ad50-ab7aa29fe142 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:49.920585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9869c8-a679-4630-ad50-ab7aa29fe142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:49.985673 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-6a9869c8-a679-4630-ad50-ab7aa29fe142 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:49.986867 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6a9869c8-a679-4630-ad50-ab7aa29fe142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:50.080183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 583/1157] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:50.373820 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d7ae6-592a-4a5b-8112-d13625a08d0c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:50.375939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8e4d7ae6-592a-4a5b-8112-d13625a08d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:50.385617 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:50.519143 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d7ae6-592a-4a5b-8112-d13625a08d0c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:50.597081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:50.629234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac3111-1cec-472d-bce9-7d569b6a26d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:50.629799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac3111-1cec-472d-bce9-7d569b6a26d8 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:50.630104 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac3111-1cec-472d-bce9-7d569b6a26d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:50.630373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac3111-1cec-472d-bce9-7d569b6a26d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:50.632006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-47ac3111-1cec-472d-bce9-7d569b6a26d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:50.718956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac3111-1cec-472d-bce9-7d569b6a26d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:50.721121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac3111-1cec-472d-bce9-7d569b6a26d8 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:50.739059 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 575/1158] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:53.080048 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b010-39e6-40c8-bb73-1bd6972bf479 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:53.080608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b010-39e6-40c8-bb73-1bd6972bf479 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:53.080608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c83b010-39e6-40c8-bb73-1bd6972bf479 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:53.081922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 584/1159] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:53.087142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab566dc-52d1-4532-bccf-8772aef7523a None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:53.087578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab566dc-52d1-4532-bccf-8772aef7523a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:53.087681 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab566dc-52d1-4532-bccf-8772aef7523a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:53.153250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-1ab566dc-52d1-4532-bccf-8772aef7523a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:53.154473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1ab566dc-52d1-4532-bccf-8772aef7523a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:53.199952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:53.249206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 576/1160] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:53.535648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83b010-39e6-40c8-bb73-1bd6972bf479 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:53.537186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7c83b010-39e6-40c8-bb73-1bd6972bf479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:53.560867 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:53.640596 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c83b010-39e6-40c8-bb73-1bd6972bf479 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:53.713971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21510056-4d8a-4349-901b-917aee017279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:53.714602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21510056-4d8a-4349-901b-917aee017279 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:53.714875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21510056-4d8a-4349-901b-917aee017279 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:53.715143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21510056-4d8a-4349-901b-917aee017279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:53.716727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-21510056-4d8a-4349-901b-917aee017279 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:53.787855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:53.821429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21510056-4d8a-4349-901b-917aee017279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:53.822883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21510056-4d8a-4349-901b-917aee017279 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:53.841492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 585/1161] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:56.022716 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596556-e336-48c7-82a9-f9937f958390 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:56.023277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596556-e336-48c7-82a9-f9937f958390 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:56.023277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20596556-e336-48c7-82a9-f9937f958390 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:56.024640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 577/1162] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:56.029816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df151e-4ca7-4999-a8d1-e4b681b1256a None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:56.030293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df151e-4ca7-4999-a8d1-e4b681b1256a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:56.030429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df151e-4ca7-4999-a8d1-e4b681b1256a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:56.095219 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-65df151e-4ca7-4999-a8d1-e4b681b1256a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:56.096411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-65df151e-4ca7-4999-a8d1-e4b681b1256a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:56.191495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 586/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:56.486440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20596556-e336-48c7-82a9-f9937f958390 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:56.487958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-20596556-e336-48c7-82a9-f9937f958390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:56.588011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20596556-e336-48c7-82a9-f9937f958390 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:56.660167 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:56.661565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c26a8-f290-49ce-b18d-028015dffe4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:56.662197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c26a8-f290-49ce-b18d-028015dffe4a service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:56.662476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c26a8-f290-49ce-b18d-028015dffe4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:56.662746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c26a8-f290-49ce-b18d-028015dffe4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:56.664321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-696c26a8-f290-49ce-b18d-028015dffe4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:56.771206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c26a8-f290-49ce-b18d-028015dffe4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:56.772636 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c26a8-f290-49ce-b18d-028015dffe4a service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:56.791117 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 578/1164] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:26:59.035605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ebba0-46d2-4fb8-b3e7-aa0b8c36e3be None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:59.036207 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ebba0-46d2-4fb8-b3e7-aa0b8c36e3be None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:59.036207 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ebba0-46d2-4fb8-b3e7-aa0b8c36e3be None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:59.037491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 587/1165] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:26:59.042647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecaacb-fa0a-4e4f-b89b-da7021f1cd35 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:59.043072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecaacb-fa0a-4e4f-b89b-da7021f1cd35 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:59.043185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecaacb-fa0a-4e4f-b89b-da7021f1cd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:59.111889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-cdecaacb-fa0a-4e4f-b89b-da7021f1cd35 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:26:59.113097 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cdecaacb-fa0a-4e4f-b89b-da7021f1cd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:59.163994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:59.207064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 579/1166] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:26:59.497550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ebba0-46d2-4fb8-b3e7-aa0b8c36e3be None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:26:59.499124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-871ebba0-46d2-4fb8-b3e7-aa0b8c36e3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:59.535061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:59.601150 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ebba0-46d2-4fb8-b3e7-aa0b8c36e3be None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:26:59.672589 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d02fac-39ad-4f70-ac51-955e7754f818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:26:59.673176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d02fac-39ad-4f70-ac51-955e7754f818 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:26:59.673446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d02fac-39ad-4f70-ac51-955e7754f818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:26:59.673706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d02fac-39ad-4f70-ac51-955e7754f818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:26:59.675295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b8d02fac-39ad-4f70-ac51-955e7754f818 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:26:59.754181 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:26:59.782498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d02fac-39ad-4f70-ac51-955e7754f818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:26:59.783998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d02fac-39ad-4f70-ac51-955e7754f818 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:26:59.801974 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 588/1167] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:26:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:27:02.246428 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d57940-600f-4e73-a498-eb1a696214ec None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:27:02.246957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d57940-600f-4e73-a498-eb1a696214ec None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:27:02.247075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d57940-600f-4e73-a498-eb1a696214ec None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:27:02.248359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 580/1168] 10.222.0.20 () {54 vars in 927 bytes} [Wed Oct 19 02:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:27:02.253563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774976cc-ea6b-4fcd-8c27-0452cae45726 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:27:02.254634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774976cc-ea6b-4fcd-8c27-0452cae45726 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:27:02.254748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774976cc-ea6b-4fcd-8c27-0452cae45726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:27:02.332052 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-774976cc-ea6b-4fcd-8c27-0452cae45726 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:27:02.333265 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-774976cc-ea6b-4fcd-8c27-0452cae45726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:27:02.428145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 589/1169] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Oct 19 02:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:27:02.713382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d57940-600f-4e73-a498-eb1a696214ec None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:27:02.714940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-94d57940-600f-4e73-a498-eb1a696214ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:27:02.814802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d57940-600f-4e73-a498-eb1a696214ec None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:27:02.885009 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44060fb-eb75-47ba-87d7-94b0e05261fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:27:02.885601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44060fb-eb75-47ba-87d7-94b0e05261fc service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:27:02.885873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44060fb-eb75-47ba-87d7-94b0e05261fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:27:02.886165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44060fb-eb75-47ba-87d7-94b0e05261fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:27:02.887732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d44060fb-eb75-47ba-87d7-94b0e05261fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:27:02.896071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:27:02.998467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44060fb-eb75-47ba-87d7-94b0e05261fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:27:02.999919 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44060fb-eb75-47ba-87d7-94b0e05261fc service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:27:03.017850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 581/1170] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:27:12.426580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=82 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:27:13.015634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=50 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:28:46.311101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65294c62-0fa9-4eb6-bbac-52f97a1d098f None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:46.311676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65294c62-0fa9-4eb6-bbac-52f97a1d098f None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:46.311676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65294c62-0fa9-4eb6-bbac-52f97a1d098f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:46.392482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-65294c62-0fa9-4eb6-bbac-52f97a1d098f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:28:46.393624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-65294c62-0fa9-4eb6-bbac-52f97a1d098f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:28:46.493273 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 590/1171] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:28:46.501248 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46df7a-b1fc-4b31-81d9-137ab68785e1 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:46.501757 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46df7a-b1fc-4b31-81d9-137ab68785e1 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:46.501863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46df7a-b1fc-4b31-81d9-137ab68785e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:46.522424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-de46df7a-b1fc-4b31-81d9-137ab68785e1 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:28:46.523601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-de46df7a-b1fc-4b31-81d9-137ab68785e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:28:46.531884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 582/1172] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:28:46.536381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65294c62-0fa9-4eb6-bbac-52f97a1d098f None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:28:46.545726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65294c62-0fa9-4eb6-bbac-52f97a1d098f None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:28:46.550297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338d847-0018-4714-872e-f25972d016c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:28:46.550888 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338d847-0018-4714-872e-f25972d016c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:46.551166 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338d847-0018-4714-872e-f25972d016c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:46.551438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338d847-0018-4714-872e-f25972d016c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:46.552077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338d847-0018-4714-872e-f25972d016c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:28:46.552374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338d847-0018-4714-872e-f25972d016c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:28:46.553382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: WARNING py.warnings [None req-f338d847-0018-4714-872e-f25972d016c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:28:46.553382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: warnings.warn(msg) Oct 19 02:28:46.553382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: Oct 19 02:28:46.554312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338d847-0018-4714-872e-f25972d016c2 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:28:46.567662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 591/1173] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:28:46.576662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46df7a-b1fc-4b31-81d9-137ab68785e1 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:28:46.586278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46df7a-b1fc-4b31-81d9-137ab68785e1 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:28:46.590847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a131d-3b34-4811-8202-3833f3fc0789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:28:46.591440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a131d-3b34-4811-8202-3833f3fc0789 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:46.591721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a131d-3b34-4811-8202-3833f3fc0789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:46.591993 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a131d-3b34-4811-8202-3833f3fc0789 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:46.592633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a131d-3b34-4811-8202-3833f3fc0789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:28:46.593765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: WARNING py.warnings [None req-2a2a131d-3b34-4811-8202-3833f3fc0789 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:28:46.593765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: warnings.warn(msg) Oct 19 02:28:46.593765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: Oct 19 02:28:46.594980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a131d-3b34-4811-8202-3833f3fc0789 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:28:46.696331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 583/1174] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:28:46 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:28:46.697633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338d847-0018-4714-872e-f25972d016c2 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:28:46.708053 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338d847-0018-4714-872e-f25972d016c2 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:28:46.755795 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:28:46.762319 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6f9aa-b86a-46b7-800b-b370f5d6b801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:28:46.762909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6f9aa-b86a-46b7-800b-b370f5d6b801 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:46.763186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6f9aa-b86a-46b7-800b-b370f5d6b801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:46.763454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6f9aa-b86a-46b7-800b-b370f5d6b801 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:46.764079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6f9aa-b86a-46b7-800b-b370f5d6b801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:28:46.765083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: WARNING py.warnings [None req-05f6f9aa-b86a-46b7-800b-b370f5d6b801 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:28:46.765083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: warnings.warn(msg) Oct 19 02:28:46.765083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: Oct 19 02:28:46.766302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f6f9aa-b86a-46b7-800b-b370f5d6b801 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:28:46.835241 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 592/1175] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:28:46 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:28:46.839248 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a131d-3b34-4811-8202-3833f3fc0789 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:28:46.850196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a131d-3b34-4811-8202-3833f3fc0789 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:28:46.854742 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2197449-c1a0-40bb-8d05-ef82dd46bdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:28:46.855366 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2197449-c1a0-40bb-8d05-ef82dd46bdf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:46.855664 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2197449-c1a0-40bb-8d05-ef82dd46bdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:46.855933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2197449-c1a0-40bb-8d05-ef82dd46bdf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:46.856601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2197449-c1a0-40bb-8d05-ef82dd46bdf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:28:46.858219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2197449-c1a0-40bb-8d05-ef82dd46bdf7 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:28:46.871589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 584/1176] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:28:46.875416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6f9aa-b86a-46b7-800b-b370f5d6b801 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:28:46.885291 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6f9aa-b86a-46b7-800b-b370f5d6b801 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:28:46.889877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b5f76-f84c-45f8-931a-4f2f98f17fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:28:46.890500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b5f76-f84c-45f8-931a-4f2f98f17fff admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:46.890779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b5f76-f84c-45f8-931a-4f2f98f17fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:46.891054 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b5f76-f84c-45f8-931a-4f2f98f17fff admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:46.891699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b5f76-f84c-45f8-931a-4f2f98f17fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:28:46.893313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b5f76-f84c-45f8-931a-4f2f98f17fff admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:28:46.907041 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 593/1177] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:28:46.911490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2197449-c1a0-40bb-8d05-ef82dd46bdf7 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:28:46.921215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2197449-c1a0-40bb-8d05-ef82dd46bdf7 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:28:46.925832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e2de3-3506-4d7d-a7b5-2f058ef760c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:28:46.926512 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e2de3-3506-4d7d-a7b5-2f058ef760c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:46.926791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e2de3-3506-4d7d-a7b5-2f058ef760c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:46.927066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e2de3-3506-4d7d-a7b5-2f058ef760c2 admin admin] PATH_INFO: `/v3/projects/1d0d202de6834b9ba66ec5fdcbab3663/users/dea35e49cefb46f3a7cad8449716e0e9/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:46.941569 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e2de3-3506-4d7d-a7b5-2f058ef760c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0d202de6834b9ba66ec5fdcbab3663, user_id=dea35e49cefb46f3a7cad8449716e0e9, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:28:46.944452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5e2de3-3506-4d7d-a7b5-2f058ef760c2 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:28:47.034981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 585/1178] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:28:46 2022] PUT /identity/v3/projects/1d0d202de6834b9ba66ec5fdcbab3663/users/dea35e49cefb46f3a7cad8449716e0e9/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:28:47.046568 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59dbf1-7048-4bfa-8892-a9fd3ad7dd81 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:47.046879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59dbf1-7048-4bfa-8892-a9fd3ad7dd81 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:47.056201 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59dbf1-7048-4bfa-8892-a9fd3ad7dd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:47.073955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-2d59dbf1-7048-4bfa-8892-a9fd3ad7dd81 None None] MFA Rules not processed for user `dea35e49cefb46f3a7cad8449716e0e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:28:47.075180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2d59dbf1-7048-4bfa-8892-a9fd3ad7dd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:28:47.137808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 594/1179] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:28:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:28:47.151469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b0ca1-25ee-49f2-aa3d-483207323f28 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:47.151758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b0ca1-25ee-49f2-aa3d-483207323f28 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:47.152030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b0ca1-25ee-49f2-aa3d-483207323f28 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:47.153349 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 586/1180] 10.222.0.20 () {54 vars in 963 bytes} [Wed Oct 19 02:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:28:47.158159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e05fa-8bbf-4dcb-a2f9-930b4688796c None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:47.158445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e05fa-8bbf-4dcb-a2f9-930b4688796c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:47.158721 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e05fa-8bbf-4dcb-a2f9-930b4688796c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:47.216319 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-704e05fa-8bbf-4dcb-a2f9-930b4688796c None None] MFA Rules not processed for user `e63c5bd8c4264db697947f025c67f512`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:28:47.217505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-704e05fa-8bbf-4dcb-a2f9-930b4688796c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:28:47.290825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 595/1181] 10.222.0.20 () {58 vars in 1060 bytes} [Wed Oct 19 02:28:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:28:47.298497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b0ca1-25ee-49f2-aa3d-483207323f28 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:28:47.308953 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b0ca1-25ee-49f2-aa3d-483207323f28 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:28:47.312659 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d183a-98ce-4944-9fec-cecb6865ceef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:28:47.313249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d183a-98ce-4944-9fec-cecb6865ceef service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:47.313522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d183a-98ce-4944-9fec-cecb6865ceef service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:47.313786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d183a-98ce-4944-9fec-cecb6865ceef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:47.322699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d183a-98ce-4944-9fec-cecb6865ceef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:28:47.323975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d183a-98ce-4944-9fec-cecb6865ceef service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:28:47.333109 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 587/1182] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:28:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:28:47.381888 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913055-293d-432d-a4c5-6f91bc7d6ad1 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:47.382344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913055-293d-432d-a4c5-6f91bc7d6ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:47.382468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1913055-293d-432d-a4c5-6f91bc7d6ad1 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:47.383751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 596/1183] 10.222.0.20 () {52 vars in 830 bytes} [Wed Oct 19 02:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:28:47.387331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d183a-98ce-4944-9fec-cecb6865ceef service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:28:47.448645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d183a-98ce-4944-9fec-cecb6865ceef service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:28:47.453263 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:28:47.453879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:47.454191 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:47.454462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] PATH_INFO: `/v3/users/dea35e49cefb46f3a7cad8449716e0e9` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:47.455585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dea35e49cefb46f3a7cad8449716e0e9)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:28:47.456628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: WARNING py.warnings [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:28:47.456628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: warnings.warn(msg) Oct 19 02:28:47.456628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: Oct 19 02:28:47.458475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:28:47.920835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dea35e49cefb46f3a7cad8449716e0e9'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:28:47.921496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dea35e49cefb46f3a7cad8449716e0e9'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:28:47.935873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dea35e49cefb46f3a7cad8449716e0e9'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:28:48.032067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dea35e49cefb46f3a7cad8449716e0e9'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:28:48.032750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dea35e49cefb46f3a7cad8449716e0e9'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:28:48.056393 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:28:48.064162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 588/1184] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:28:47 2022] DELETE /identity/v3/users/dea35e49cefb46f3a7cad8449716e0e9 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:28:48.072982 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e960d0-6abc-4b05-bda2-ac4707d3b442 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:48.073261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e960d0-6abc-4b05-bda2-ac4707d3b442 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:48.073535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e960d0-6abc-4b05-bda2-ac4707d3b442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:48.093181 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-92e960d0-6abc-4b05-bda2-ac4707d3b442 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:28:48.094381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-92e960d0-6abc-4b05-bda2-ac4707d3b442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:28:48.104185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:28:48.153226 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 597/1185] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:28:48.163805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:28:48.165245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:28:48.242239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116fa65-1dfd-427f-bbf9-49d0a8e16889 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:28:48.245806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3755e-3664-4476-834d-6d1276e7e8b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:28:48.246434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3755e-3664-4476-834d-6d1276e7e8b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:48.246712 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3755e-3664-4476-834d-6d1276e7e8b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:48.246981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3755e-3664-4476-834d-6d1276e7e8b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:48.256097 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3755e-3664-4476-834d-6d1276e7e8b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:28:48.257549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3755e-3664-4476-834d-6d1276e7e8b1 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:28:48.267351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 589/1186] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:28:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:28:48.748645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e960d0-6abc-4b05-bda2-ac4707d3b442 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:28:48.750165 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-92e960d0-6abc-4b05-bda2-ac4707d3b442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:28:48.762669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e960d0-6abc-4b05-bda2-ac4707d3b442 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:28:48.768937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:28:48.769661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:28:48.769944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:28:48.770255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] PATH_INFO: `/v3/projects/1d0d202de6834b9ba66ec5fdcbab3663` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:28:48.771467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d0d202de6834b9ba66ec5fdcbab3663)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:28:48.773276 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: WARNING py.warnings [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:28:48.773276 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: warnings.warn(msg) Oct 19 02:28:48.773276 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: Oct 19 02:28:48.774872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:28:48.892894 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d0d202de6834b9ba66ec5fdcbab3663', 'request_id': 'req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:28:48.907807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d0d202de6834b9ba66ec5fdcbab3663', 'request_id': 'req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:28:49.001501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d0d202de6834b9ba66ec5fdcbab3663', 'request_id': 'req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:28:49.013596 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] The token cache is being invalidate because project 1d0d202de6834b9ba66ec5fdcbab3663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:28:49.013885 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:28:49.014642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-87095aa1-c8c9-4d62-ab8e-9fe8f2f463c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:28:49.016156 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 598/1187] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:28:48 2022] DELETE /identity/v3/projects/1d0d202de6834b9ba66ec5fdcbab3663 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:28:58.263334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:28:59.010427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:30:18.442712 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188f3a5-9ca3-4caa-8378-59d3c30e5564 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:18.443299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188f3a5-9ca3-4caa-8378-59d3c30e5564 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:18.443299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188f3a5-9ca3-4caa-8378-59d3c30e5564 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:18.444536 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 590/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:30:18.449508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f7cf3-b102-4836-9e4c-71fe391ecc7c None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:18.449942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f7cf3-b102-4836-9e4c-71fe391ecc7c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:18.450073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f7cf3-b102-4836-9e4c-71fe391ecc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:18.469978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-062f7cf3-b102-4836-9e4c-71fe391ecc7c None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:30:18.471194 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-062f7cf3-b102-4836-9e4c-71fe391ecc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:18.527112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 599/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:30:18.679984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188f3a5-9ca3-4caa-8378-59d3c30e5564 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:30:18.684866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7188f3a5-9ca3-4caa-8378-59d3c30e5564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:18.781512 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188f3a5-9ca3-4caa-8378-59d3c30e5564 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:30:18.787046 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f4a95-d059-4469-b3d3-183f11da9e67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:30:18.787645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f4a95-d059-4469-b3d3-183f11da9e67 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:18.787920 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f4a95-d059-4469-b3d3-183f11da9e67 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:18.788185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f4a95-d059-4469-b3d3-183f11da9e67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:18.796289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f4a95-d059-4469-b3d3-183f11da9e67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:30:18.797886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f4a95-d059-4469-b3d3-183f11da9e67 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:30:18.807687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 591/1190] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:30:19.898140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3420f-d268-4cf3-94b0-2733cc2c8032 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:19.898701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3420f-d268-4cf3-94b0-2733cc2c8032 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:19.898701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3420f-d268-4cf3-94b0-2733cc2c8032 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:19.900139 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 600/1191] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:30:19.907189 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64383e-cfc1-4574-8e13-beacf64d5fcc None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:19.907634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64383e-cfc1-4574-8e13-beacf64d5fcc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:19.907746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64383e-cfc1-4574-8e13-beacf64d5fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:19.930568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-5d64383e-cfc1-4574-8e13-beacf64d5fcc None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:30:19.931848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5d64383e-cfc1-4574-8e13-beacf64d5fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:19.940013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 592/1192] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:30:20.094831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3420f-d268-4cf3-94b0-2733cc2c8032 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:30:20.109642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3420f-d268-4cf3-94b0-2733cc2c8032 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:30:20.115147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5296a9-23f5-4828-9112-e418f405a271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:30:20.115750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5296a9-23f5-4828-9112-e418f405a271 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:20.116028 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5296a9-23f5-4828-9112-e418f405a271 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:20.116302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5296a9-23f5-4828-9112-e418f405a271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:20.124556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5296a9-23f5-4828-9112-e418f405a271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:30:20.126188 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5296a9-23f5-4828-9112-e418f405a271 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:30:20.135376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 601/1193] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:30:20.456758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300b699-92a0-441a-9047-3ae4780dc2f4 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:20.456968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300b699-92a0-441a-9047-3ae4780dc2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:20.457244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300b699-92a0-441a-9047-3ae4780dc2f4 None None] PATH_INFO: `/v3` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:20.458568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 593/1194] 10.222.0.13 () {54 vars in 925 bytes} [Wed Oct 19 02:30:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:30:20.477598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0282d-65a3-4471-9bc4-ff83482e83e1 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:20.477887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0282d-65a3-4471-9bc4-ff83482e83e1 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:20.478191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a0282d-65a3-4471-9bc4-ff83482e83e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:20.526707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-f4a0282d-65a3-4471-9bc4-ff83482e83e1 None None] MFA Rules not processed for user `9243ebe24ae84157b74696c29bda3279`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:30:20.527895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f4a0282d-65a3-4471-9bc4-ff83482e83e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:20.595849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 602/1195] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Oct 19 02:30:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:30:24.404136 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866d925-bc0e-4f16-afe2-dd5d139eef57 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:24.404560 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866d925-bc0e-4f16-afe2-dd5d139eef57 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:24.404660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866d925-bc0e-4f16-afe2-dd5d139eef57 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:24.406012 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 594/1196] 10.222.0.20 () {54 vars in 961 bytes} [Wed Oct 19 02:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:30:24.411559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a3a70-b09d-4b3f-9aba-09b16ef0edba None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:24.412027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a3a70-b09d-4b3f-9aba-09b16ef0edba None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:24.412212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a3a70-b09d-4b3f-9aba-09b16ef0edba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:24.431619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-733a3a70-b09d-4b3f-9aba-09b16ef0edba None None] MFA Rules not processed for user `9243ebe24ae84157b74696c29bda3279`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:30:24.433014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-733a3a70-b09d-4b3f-9aba-09b16ef0edba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:24.441549 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 603/1197] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Oct 19 02:30:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:30:24.449008 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866d925-bc0e-4f16-afe2-dd5d139eef57 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:30:24.462951 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866d925-bc0e-4f16-afe2-dd5d139eef57 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:30:24.467840 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241b6e5-b87c-43f7-abfa-6dd048bed1c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:30:24.468431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241b6e5-b87c-43f7-abfa-6dd048bed1c3 service placement] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:24.468707 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241b6e5-b87c-43f7-abfa-6dd048bed1c3 service placement] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:24.468976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241b6e5-b87c-43f7-abfa-6dd048bed1c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:24.477696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241b6e5-b87c-43f7-abfa-6dd048bed1c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:30:24.479296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5241b6e5-b87c-43f7-abfa-6dd048bed1c3 service placement] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:30:24.488575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 595/1198] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:30:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:30:31.615642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cfe38-dac7-40f8-b3ad-e8f0b9570fa4 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:31.616202 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cfe38-dac7-40f8-b3ad-e8f0b9570fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:31.616202 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cfe38-dac7-40f8-b3ad-e8f0b9570fa4 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:31.617553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 604/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:30:31.623890 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0413984-e61e-42c0-ae39-eb897e7f9489 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:31.624320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0413984-e61e-42c0-ae39-eb897e7f9489 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:31.624445 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0413984-e61e-42c0-ae39-eb897e7f9489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:31.643909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-e0413984-e61e-42c0-ae39-eb897e7f9489 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:30:31.645103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e0413984-e61e-42c0-ae39-eb897e7f9489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:31.652915 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 596/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:30:31.810010 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cfe38-dac7-40f8-b3ad-e8f0b9570fa4 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:30:31.824082 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cfe38-dac7-40f8-b3ad-e8f0b9570fa4 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:30:31.829410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed386b-4398-4624-ba21-d384c6e22702 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:30:31.830538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed386b-4398-4624-ba21-d384c6e22702 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:31.830820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed386b-4398-4624-ba21-d384c6e22702 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:31.831092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed386b-4398-4624-ba21-d384c6e22702 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:31.839030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed386b-4398-4624-ba21-d384c6e22702 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:30:31.840611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed386b-4398-4624-ba21-d384c6e22702 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:30:31.850475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 605/1201] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:30:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:30:32.920722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54cbbe-cfdd-41f4-8904-c1ce1f6d8e73 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:32.921333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54cbbe-cfdd-41f4-8904-c1ce1f6d8e73 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:32.921333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54cbbe-cfdd-41f4-8904-c1ce1f6d8e73 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:32.922631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 597/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 19 02:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:30:32.928929 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939eba0-8a94-4060-9d54-7f13a48958c2 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:32.929388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939eba0-8a94-4060-9d54-7f13a48958c2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:32.929584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939eba0-8a94-4060-9d54-7f13a48958c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:32.949797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8939eba0-8a94-4060-9d54-7f13a48958c2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:30:32.955825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8939eba0-8a94-4060-9d54-7f13a48958c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:32.959382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 606/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 19 02:30:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:30:33.110566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54cbbe-cfdd-41f4-8904-c1ce1f6d8e73 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:30:33.125514 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe54cbbe-cfdd-41f4-8904-c1ce1f6d8e73 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:30:33.131642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45803930-ffb8-4568-b947-8df6635d3253 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:30:33.132276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45803930-ffb8-4568-b947-8df6635d3253 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:33.132554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45803930-ffb8-4568-b947-8df6635d3253 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:33.132823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45803930-ffb8-4568-b947-8df6635d3253 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:33.140851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45803930-ffb8-4568-b947-8df6635d3253 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:30:33.142457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45803930-ffb8-4568-b947-8df6635d3253 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:30:33.151767 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 598/1204] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:30:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:30:37.103539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c70c7a-a451-4c34-b3ce-971805e63baa None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:37.104114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c70c7a-a451-4c34-b3ce-971805e63baa None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:37.104114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c70c7a-a451-4c34-b3ce-971805e63baa None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:37.105414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 607/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:30:37.110817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c6e0-105f-46e0-9e87-a09dabad68b0 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:37.111241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c6e0-105f-46e0-9e87-a09dabad68b0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:37.111382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c6e0-105f-46e0-9e87-a09dabad68b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:37.131523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c677c6e0-105f-46e0-9e87-a09dabad68b0 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:30:37.132694 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c677c6e0-105f-46e0-9e87-a09dabad68b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:37.140279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 599/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:30:37.436244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c70c7a-a451-4c34-b3ce-971805e63baa None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:30:37.437901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d8c70c7a-a451-4c34-b3ce-971805e63baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:37.465249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:30:37.494176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c70c7a-a451-4c34-b3ce-971805e63baa None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:30:37.497975 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba879c-172f-45ed-95be-a6b02aac1a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:30:37.498602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba879c-172f-45ed-95be-a6b02aac1a96 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:37.498883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba879c-172f-45ed-95be-a6b02aac1a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:37.499159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba879c-172f-45ed-95be-a6b02aac1a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:37.508283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba879c-172f-45ed-95be-a6b02aac1a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:30:37.509760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba879c-172f-45ed-95be-a6b02aac1a96 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:30:37.519336 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 608/1207] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:30:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:30:39.702600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980e3f5-04d8-4b2f-af5c-c67b80464fbc None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:39.703179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980e3f5-04d8-4b2f-af5c-c67b80464fbc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:39.703179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980e3f5-04d8-4b2f-af5c-c67b80464fbc None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:39.704504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 600/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:30:39.710195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec21b19-59c1-4360-a15b-aa588a010da2 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:39.710621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec21b19-59c1-4360-a15b-aa588a010da2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:39.710746 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec21b19-59c1-4360-a15b-aa588a010da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:39.729777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-cec21b19-59c1-4360-a15b-aa588a010da2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:30:39.731005 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cec21b19-59c1-4360-a15b-aa588a010da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:39.738737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 609/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:30:40.023378 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980e3f5-04d8-4b2f-af5c-c67b80464fbc None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:30:40.025073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4980e3f5-04d8-4b2f-af5c-c67b80464fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:40.038163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980e3f5-04d8-4b2f-af5c-c67b80464fbc None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:30:40.041518 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5460e0a-a52c-4e0b-87e7-cc3a37ef78f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:30:40.042131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460e0a-a52c-4e0b-87e7-cc3a37ef78f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:40.042406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460e0a-a52c-4e0b-87e7-cc3a37ef78f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:40.042672 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460e0a-a52c-4e0b-87e7-cc3a37ef78f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:40.052711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5460e0a-a52c-4e0b-87e7-cc3a37ef78f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:30:40.054205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5460e0a-a52c-4e0b-87e7-cc3a37ef78f8 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:30:40.063933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 601/1210] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:30:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:30:43.692242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9bb7b-ad36-486e-bf91-4ca2b182c6b6 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:43.692800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9bb7b-ad36-486e-bf91-4ca2b182c6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:43.692800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9bb7b-ad36-486e-bf91-4ca2b182c6b6 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:43.694161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 610/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:30:43.699728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da15ed-a18f-4be4-8da4-0183d60a016a None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:43.700243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da15ed-a18f-4be4-8da4-0183d60a016a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:43.700347 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da15ed-a18f-4be4-8da4-0183d60a016a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:43.720742 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c6da15ed-a18f-4be4-8da4-0183d60a016a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:30:43.722013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c6da15ed-a18f-4be4-8da4-0183d60a016a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:43.730289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 602/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:30:44.036176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9bb7b-ad36-486e-bf91-4ca2b182c6b6 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:30:44.037963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a2f9bb7b-ad36-486e-bf91-4ca2b182c6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:44.098019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9bb7b-ad36-486e-bf91-4ca2b182c6b6 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:30:44.103458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c20b0a-9368-47b4-a97d-c3c5410091f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:30:44.104073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20b0a-9368-47b4-a97d-c3c5410091f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:44.104406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20b0a-9368-47b4-a97d-c3c5410091f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:44.104621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20b0a-9368-47b4-a97d-c3c5410091f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:44.114785 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c20b0a-9368-47b4-a97d-c3c5410091f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:30:44.116307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c20b0a-9368-47b4-a97d-c3c5410091f5 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:30:44.126623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 611/1213] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:30:46.311556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe4c7c-5a26-464c-a626-418ba118c7c9 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:46.312071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe4c7c-5a26-464c-a626-418ba118c7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:46.312189 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe4c7c-5a26-464c-a626-418ba118c7c9 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:46.313451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 603/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 19 02:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:30:46.319010 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d35eb-97b2-4e0a-a628-60e0b43f341c None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:46.319495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d35eb-97b2-4e0a-a628-60e0b43f341c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:46.319601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d35eb-97b2-4e0a-a628-60e0b43f341c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:46.340807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-be3d35eb-97b2-4e0a-a628-60e0b43f341c None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:30:46.341991 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-be3d35eb-97b2-4e0a-a628-60e0b43f341c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:46.350695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 612/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 19 02:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:30:46.651701 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe4c7c-5a26-464c-a626-418ba118c7c9 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:30:46.653362 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0bfe4c7c-5a26-464c-a626-418ba118c7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:30:46.667305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe4c7c-5a26-464c-a626-418ba118c7c9 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:30:46.671134 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed73f0-124f-40dd-80cf-8a01d0e66cb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:30:46.671740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed73f0-124f-40dd-80cf-8a01d0e66cb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:30:46.672019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed73f0-124f-40dd-80cf-8a01d0e66cb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:30:46.672294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed73f0-124f-40dd-80cf-8a01d0e66cb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:30:46.684467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed73f0-124f-40dd-80cf-8a01d0e66cb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:30:46.685893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed73f0-124f-40dd-80cf-8a01d0e66cb7 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:30:46.695452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 604/1216] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:30:56.335824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=47 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:30:56.691071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=82 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:11.701337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77029c7b-8a2b-4ec0-b283-b1f8934b8ef8 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:11.703990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77029c7b-8a2b-4ec0-b283-b1f8934b8ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:11.703990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77029c7b-8a2b-4ec0-b283-b1f8934b8ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:11.724472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-77029c7b-8a2b-4ec0-b283-b1f8934b8ef8 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:11.726418 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-77029c7b-8a2b-4ec0-b283-b1f8934b8ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:11.738347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 613/1217] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:11.749376 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa93d9e-a261-4d01-b7e9-621fd112e5cc None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:11.749968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa93d9e-a261-4d01-b7e9-621fd112e5cc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:11.749968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa93d9e-a261-4d01-b7e9-621fd112e5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:11.779136 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-6aa93d9e-a261-4d01-b7e9-621fd112e5cc None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:11.780341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6aa93d9e-a261-4d01-b7e9-621fd112e5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:11.788887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 605/1218] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:11.793960 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77029c7b-8a2b-4ec0-b283-b1f8934b8ef8 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:11.810167 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77029c7b-8a2b-4ec0-b283-b1f8934b8ef8 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:11.815341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c667dc-137a-426a-8e5a-28fa0ca05477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:11.816202 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c667dc-137a-426a-8e5a-28fa0ca05477 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:11.816835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c667dc-137a-426a-8e5a-28fa0ca05477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:11.817409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c667dc-137a-426a-8e5a-28fa0ca05477 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:11.818294 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c667dc-137a-426a-8e5a-28fa0ca05477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:11.818818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c667dc-137a-426a-8e5a-28fa0ca05477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:11.820627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c667dc-137a-426a-8e5a-28fa0ca05477 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:11.836983 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 614/1219] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:11.845667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa93d9e-a261-4d01-b7e9-621fd112e5cc None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:11.861475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa93d9e-a261-4d01-b7e9-621fd112e5cc None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:11.876862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd61b27-4909-4199-851b-64940e3945f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:11.877499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd61b27-4909-4199-851b-64940e3945f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:11.877780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd61b27-4909-4199-851b-64940e3945f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:11.878073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd61b27-4909-4199-851b-64940e3945f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:11.878735 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd61b27-4909-4199-851b-64940e3945f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:11.880889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd61b27-4909-4199-851b-64940e3945f3 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:11.941006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eff13-1089-4e8b-9368-844d2ab59126 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:11.941830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eff13-1089-4e8b-9368-844d2ab59126 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:11.942362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eff13-1089-4e8b-9368-844d2ab59126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:11.969669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-9d5eff13-1089-4e8b-9368-844d2ab59126 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:11.971568 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9d5eff13-1089-4e8b-9368-844d2ab59126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:11.987146 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 615/1220] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:11.997764 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710c5fe-698f-44c9-9067-3b6a5cf2df6e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:11.998460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710c5fe-698f-44c9-9067-3b6a5cf2df6e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:11.999225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710c5fe-698f-44c9-9067-3b6a5cf2df6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.017127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 606/1221] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:11 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:12.021608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd61b27-4909-4199-851b-64940e3945f3 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.024312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-5710c5fe-698f-44c9-9067-3b6a5cf2df6e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.025592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5710c5fe-698f-44c9-9067-3b6a5cf2df6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.132272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 616/1222] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:12.138547 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd8bc4-c50b-4109-878b-29e82f88061a None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.139021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd8bc4-c50b-4109-878b-29e82f88061a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.139517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd8bc4-c50b-4109-878b-29e82f88061a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.150497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd61b27-4909-4199-851b-64940e3945f3 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.164956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee353415-7aa9-4165-b970-a2ae8aca53d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.165636 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee353415-7aa9-4165-b970-a2ae8aca53d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.165912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee353415-7aa9-4165-b970-a2ae8aca53d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.166211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee353415-7aa9-4165-b970-a2ae8aca53d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.166844 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee353415-7aa9-4165-b970-a2ae8aca53d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.168128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: WARNING py.warnings [None req-ee353415-7aa9-4165-b970-a2ae8aca53d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:32:12.168128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: warnings.warn(msg) Oct 19 02:32:12.168128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: Oct 19 02:32:12.169961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee353415-7aa9-4165-b970-a2ae8aca53d9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.172312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-0fbd8bc4-c50b-4109-878b-29e82f88061a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.173533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0fbd8bc4-c50b-4109-878b-29e82f88061a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.190310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 617/1223] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.191586 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cb5db-a9b0-4a86-bdae-fe7f718e7a17 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.191873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cb5db-a9b0-4a86-bdae-fe7f718e7a17 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.192145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cb5db-a9b0-4a86-bdae-fe7f718e7a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.217922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-0e0cb5db-a9b0-4a86-bdae-fe7f718e7a17 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.219141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0e0cb5db-a9b0-4a86-bdae-fe7f718e7a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.227621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 618/1224] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.228897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdd85d-aea6-4f1b-975b-e96bda9e4019 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.229183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdd85d-aea6-4f1b-975b-e96bda9e4019 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.229454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdd85d-aea6-4f1b-975b-e96bda9e4019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.252060 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-72bdd85d-aea6-4f1b-975b-e96bda9e4019 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.253293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-72bdd85d-aea6-4f1b-975b-e96bda9e4019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.261158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 619/1225] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.262903 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdd85d-aea6-4f1b-975b-e96bda9e4019 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.266510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 607/1226] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:32:12.267977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0602c1-e687-4f25-9331-6aea0b35ed68 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.268332 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0602c1-e687-4f25-9331-6aea0b35ed68 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.268675 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0602c1-e687-4f25-9331-6aea0b35ed68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.272758 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdd85d-aea6-4f1b-975b-e96bda9e4019 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.277272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0137a-0fb5-4216-8fef-bbfb023d1671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.277893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0137a-0fb5-4216-8fef-bbfb023d1671 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.278211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0137a-0fb5-4216-8fef-bbfb023d1671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.278475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0137a-0fb5-4216-8fef-bbfb023d1671 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.279112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0137a-0fb5-4216-8fef-bbfb023d1671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:12.279409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0137a-0fb5-4216-8fef-bbfb023d1671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.280994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0137a-0fb5-4216-8fef-bbfb023d1671 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.290770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-4a0602c1-e687-4f25-9331-6aea0b35ed68 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.291972 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4a0602c1-e687-4f25-9331-6aea0b35ed68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.292356 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 620/1227] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:12.295472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb567e21-55b7-4ffe-be8f-73a1537991bf None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.295895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb567e21-55b7-4ffe-be8f-73a1537991bf None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.296394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb567e21-55b7-4ffe-be8f-73a1537991bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.304972 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 608/1228] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.304972 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014497-fa7a-4264-b063-4602f5d6283f None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.304972 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014497-fa7a-4264-b063-4602f5d6283f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.304972 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014497-fa7a-4264-b063-4602f5d6283f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.321073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-bb567e21-55b7-4ffe-be8f-73a1537991bf None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.322314 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-bb567e21-55b7-4ffe-be8f-73a1537991bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.322958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-4e014497-fa7a-4264-b063-4602f5d6283f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.324117 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4e014497-fa7a-4264-b063-4602f5d6283f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.330144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 621/1229] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.331273 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9085c6-a905-44e5-ac03-d059b64028ff None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.331559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9085c6-a905-44e5-ac03-d059b64028ff None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.331831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9085c6-a905-44e5-ac03-d059b64028ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.338214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 609/1230] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.338214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a6491-cc37-42fe-b3b0-858d507aba38 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.338214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a6491-cc37-42fe-b3b0-858d507aba38 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.338214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a6491-cc37-42fe-b3b0-858d507aba38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.355257 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-7d9085c6-a905-44e5-ac03-d059b64028ff None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.357009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7d9085c6-a905-44e5-ac03-d059b64028ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.357155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-072a6491-cc37-42fe-b3b0-858d507aba38 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.358192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-072a6491-cc37-42fe-b3b0-858d507aba38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.364718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 622/1231] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.365481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9085c6-a905-44e5-ac03-d059b64028ff None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.367285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 610/1232] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.368876 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e50945-eb03-4dc2-ada4-d2c37ff3d844 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.369163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e50945-eb03-4dc2-ada4-d2c37ff3d844 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.369467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e50945-eb03-4dc2-ada4-d2c37ff3d844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.375129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9085c6-a905-44e5-ac03-d059b64028ff None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.379704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0724c-11b3-4357-84c5-fc35b3891d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.380310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0724c-11b3-4357-84c5-fc35b3891d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.380591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0724c-11b3-4357-84c5-fc35b3891d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.380861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff0724c-11b3-4357-84c5-fc35b3891d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.381501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0724c-11b3-4357-84c5-fc35b3891d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.383136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff0724c-11b3-4357-84c5-fc35b3891d41 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.389913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-b2e50945-eb03-4dc2-ada4-d2c37ff3d844 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.391165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b2e50945-eb03-4dc2-ada4-d2c37ff3d844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.395927 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 623/1233] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:12.395927 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0724c-11b3-4357-84c5-fc35b3891d41 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.399118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 611/1234] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.400002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e50945-eb03-4dc2-ada4-d2c37ff3d844 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.403766 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff0724c-11b3-4357-84c5-fc35b3891d41 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.408392 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba15844-a529-42e5-a424-facc79535c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.408977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba15844-a529-42e5-a424-facc79535c24 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.409256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba15844-a529-42e5-a424-facc79535c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.409529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba15844-a529-42e5-a424-facc79535c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.410189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba15844-a529-42e5-a424-facc79535c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.411198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: WARNING py.warnings [None req-8ba15844-a529-42e5-a424-facc79535c24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:32:12.411198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: warnings.warn(msg) Oct 19 02:32:12.411198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: Oct 19 02:32:12.412069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e50945-eb03-4dc2-ada4-d2c37ff3d844 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.412383 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba15844-a529-42e5-a424-facc79535c24 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.416540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33ee7a-6806-44c6-a33b-bfd0c301e8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.417130 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33ee7a-6806-44c6-a33b-bfd0c301e8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.417407 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33ee7a-6806-44c6-a33b-bfd0c301e8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.417892 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33ee7a-6806-44c6-a33b-bfd0c301e8d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.418924 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33ee7a-6806-44c6-a33b-bfd0c301e8d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:12.419230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33ee7a-6806-44c6-a33b-bfd0c301e8d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.420223 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: WARNING py.warnings [None req-dc33ee7a-6806-44c6-a33b-bfd0c301e8d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:32:12.420223 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: warnings.warn(msg) Oct 19 02:32:12.420223 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: Oct 19 02:32:12.421087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc33ee7a-6806-44c6-a33b-bfd0c301e8d9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.432638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 612/1235] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:12.433913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc0ce-ace3-4297-9e06-79d9c49284d7 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.434224 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc0ce-ace3-4297-9e06-79d9c49284d7 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.434499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cc0ce-ace3-4297-9e06-79d9c49284d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.453403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ca5cc0ce-ace3-4297-9e06-79d9c49284d7 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.454614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ca5cc0ce-ace3-4297-9e06-79d9c49284d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.463108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 613/1236] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.464553 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6e65-54dd-40da-830f-a19deb02ecad None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.464834 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6e65-54dd-40da-830f-a19deb02ecad None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.465646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e6e65-54dd-40da-830f-a19deb02ecad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.485921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-274e6e65-54dd-40da-830f-a19deb02ecad None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.487059 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-274e6e65-54dd-40da-830f-a19deb02ecad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.494913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 614/1237] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.495662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e6e65-54dd-40da-830f-a19deb02ecad None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.505381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e6e65-54dd-40da-830f-a19deb02ecad None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.512027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 624/1238] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:12.512027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6426ea-4c0e-4287-8129-9451f33125ae None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.512269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19304f-10d1-4f30-b1fa-d1d2e3c10f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.512388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6426ea-4c0e-4287-8129-9451f33125ae None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.512388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6426ea-4c0e-4287-8129-9451f33125ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.512589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19304f-10d1-4f30-b1fa-d1d2e3c10f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.512589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19304f-10d1-4f30-b1fa-d1d2e3c10f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.512589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19304f-10d1-4f30-b1fa-d1d2e3c10f8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.512589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19304f-10d1-4f30-b1fa-d1d2e3c10f8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:12.512589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19304f-10d1-4f30-b1fa-d1d2e3c10f8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.514067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19304f-10d1-4f30-b1fa-d1d2e3c10f8d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.527145 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 615/1239] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:12.527145 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19304f-10d1-4f30-b1fa-d1d2e3c10f8d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.537668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-4e6426ea-4c0e-4287-8129-9451f33125ae None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:12.537668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4e6426ea-4c0e-4287-8129-9451f33125ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:12.537891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19304f-10d1-4f30-b1fa-d1d2e3c10f8d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.583220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:12.585121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:12.592061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f2feb-770e-4e4f-bec9-4dce4cc71212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.592700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f2feb-770e-4e4f-bec9-4dce4cc71212 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.592976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f2feb-770e-4e4f-bec9-4dce4cc71212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.593247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f2feb-770e-4e4f-bec9-4dce4cc71212 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.593880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f2feb-770e-4e4f-bec9-4dce4cc71212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.598220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 625/1240] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:12.598220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6426ea-4c0e-4287-8129-9451f33125ae None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.598498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f2feb-770e-4e4f-bec9-4dce4cc71212 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.604199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6426ea-4c0e-4287-8129-9451f33125ae None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.605647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 616/1241] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:12.607037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f2feb-770e-4e4f-bec9-4dce4cc71212 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.612574 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253bf0a-b2f2-43d4-a306-9a3ac5e3c122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.613492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253bf0a-b2f2-43d4-a306-9a3ac5e3c122 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.613492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253bf0a-b2f2-43d4-a306-9a3ac5e3c122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.613758 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253bf0a-b2f2-43d4-a306-9a3ac5e3c122 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.614342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253bf0a-b2f2-43d4-a306-9a3ac5e3c122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:12.614640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253bf0a-b2f2-43d4-a306-9a3ac5e3c122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.616224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253bf0a-b2f2-43d4-a306-9a3ac5e3c122 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.618026 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f2feb-770e-4e4f-bec9-4dce4cc71212 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.622909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c010c8-6c2a-46d5-81ef-4a53dc21b028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.623512 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c010c8-6c2a-46d5-81ef-4a53dc21b028 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.624042 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c010c8-6c2a-46d5-81ef-4a53dc21b028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.624318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c010c8-6c2a-46d5-81ef-4a53dc21b028 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.625358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c010c8-6c2a-46d5-81ef-4a53dc21b028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.626886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c010c8-6c2a-46d5-81ef-4a53dc21b028 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.635589 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 626/1242] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:12.635589 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253bf0a-b2f2-43d4-a306-9a3ac5e3c122 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.644981 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253bf0a-b2f2-43d4-a306-9a3ac5e3c122 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.649584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe6d7d-c413-46f8-a191-6d16c045bfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.650247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe6d7d-c413-46f8-a191-6d16c045bfa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.650530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe6d7d-c413-46f8-a191-6d16c045bfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.650802 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe6d7d-c413-46f8-a191-6d16c045bfa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.651445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe6d7d-c413-46f8-a191-6d16c045bfa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:12.651750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe6d7d-c413-46f8-a191-6d16c045bfa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.653869 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe6d7d-c413-46f8-a191-6d16c045bfa7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.665624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 627/1243] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:12.666436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe6d7d-c413-46f8-a191-6d16c045bfa7 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.675729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe6d7d-c413-46f8-a191-6d16c045bfa7 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.680237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e73bf-1b43-4d03-a28f-2fb5bb0b2c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.680802 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e73bf-1b43-4d03-a28f-2fb5bb0b2c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.681080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e73bf-1b43-4d03-a28f-2fb5bb0b2c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.681351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e73bf-1b43-4d03-a28f-2fb5bb0b2c8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.681971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e73bf-1b43-4d03-a28f-2fb5bb0b2c8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:12.682318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e73bf-1b43-4d03-a28f-2fb5bb0b2c8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.684400 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e73bf-1b43-4d03-a28f-2fb5bb0b2c8b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.696464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 628/1244] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:12.696464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e73bf-1b43-4d03-a28f-2fb5bb0b2c8b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.705629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e73bf-1b43-4d03-a28f-2fb5bb0b2c8b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.710192 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef0956-6955-4b73-be87-dbfae64617bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.710750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef0956-6955-4b73-be87-dbfae64617bf admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.711029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef0956-6955-4b73-be87-dbfae64617bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.711299 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef0956-6955-4b73-be87-dbfae64617bf admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.711905 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef0956-6955-4b73-be87-dbfae64617bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.713775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef0956-6955-4b73-be87-dbfae64617bf admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.764649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 617/1245] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:12.765604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c010c8-6c2a-46d5-81ef-4a53dc21b028 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.813890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 629/1246] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:12.815008 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef0956-6955-4b73-be87-dbfae64617bf admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.825552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c010c8-6c2a-46d5-81ef-4a53dc21b028 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.825858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef0956-6955-4b73-be87-dbfae64617bf admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.830185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1445a5-4f00-466f-99cc-6a2a6381254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.830805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1445a5-4f00-466f-99cc-6a2a6381254e admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.831159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571037c-4d92-4681-b9dc-e831b82f1cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.831374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1445a5-4f00-466f-99cc-6a2a6381254e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.831757 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571037c-4d92-4681-b9dc-e831b82f1cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.831885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1445a5-4f00-466f-99cc-6a2a6381254e admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.832037 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571037c-4d92-4681-b9dc-e831b82f1cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.832310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571037c-4d92-4681-b9dc-e831b82f1cd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.832446 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1445a5-4f00-466f-99cc-6a2a6381254e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.832990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571037c-4d92-4681-b9dc-e831b82f1cd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:12.833294 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571037c-4d92-4681-b9dc-e831b82f1cd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.834956 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571037c-4d92-4681-b9dc-e831b82f1cd0 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.835081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1445a5-4f00-466f-99cc-6a2a6381254e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.847519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 630/1247] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:12.848304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571037c-4d92-4681-b9dc-e831b82f1cd0 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.859146 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571037c-4d92-4681-b9dc-e831b82f1cd0 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:12.863692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4562ec9-749d-41bf-9c73-b2bd1e1448d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:12.864322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562ec9-749d-41bf-9c73-b2bd1e1448d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:12.864600 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562ec9-749d-41bf-9c73-b2bd1e1448d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:12.864868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4562ec9-749d-41bf-9c73-b2bd1e1448d5 admin admin] PATH_INFO: `/v3/projects/765785515cdb43c49525c3d1cfa8be6a/users/75e5de4fccd243e580a12e2475162451/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:12.879729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4562ec9-749d-41bf-9c73-b2bd1e1448d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=765785515cdb43c49525c3d1cfa8be6a, user_id=75e5de4fccd243e580a12e2475162451, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:12.882451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4562ec9-749d-41bf-9c73-b2bd1e1448d5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:12.907762 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 618/1248] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/projects => generated 369 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:12.910807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1445a5-4f00-466f-99cc-6a2a6381254e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.952410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 631/1249] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:12 2022] PUT /identity/v3/projects/765785515cdb43c49525c3d1cfa8be6a/users/75e5de4fccd243e580a12e2475162451/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:12.953218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4562ec9-749d-41bf-9c73-b2bd1e1448d5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:12.972405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1445a5-4f00-466f-99cc-6a2a6381254e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.014160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4562ec9-749d-41bf-9c73-b2bd1e1448d5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.018730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81086a93-b17a-45db-bd92-b1b0a6ff3bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.019368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81086a93-b17a-45db-bd92-b1b0a6ff3bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.019643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81086a93-b17a-45db-bd92-b1b0a6ff3bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.019912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81086a93-b17a-45db-bd92-b1b0a6ff3bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.020533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81086a93-b17a-45db-bd92-b1b0a6ff3bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.022459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81086a93-b17a-45db-bd92-b1b0a6ff3bd6 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.029411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b75c3-8c01-4f83-b535-2318d2ee54fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.030020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b75c3-8c01-4f83-b535-2318d2ee54fa admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.030333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b75c3-8c01-4f83-b535-2318d2ee54fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.030606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b75c3-8c01-4f83-b535-2318d2ee54fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.031240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b75c3-8c01-4f83-b535-2318d2ee54fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.033141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b75c3-8c01-4f83-b535-2318d2ee54fa admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.038418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:13.118933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 632/1250] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:13.119946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81086a93-b17a-45db-bd92-b1b0a6ff3bd6 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.148284 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 619/1251] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:12 2022] POST /identity/v3/projects => generated 383 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:13.149032 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b75c3-8c01-4f83-b535-2318d2ee54fa admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.168047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:13.181206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81086a93-b17a-45db-bd92-b1b0a6ff3bd6 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.209475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b75c3-8c01-4f83-b535-2318d2ee54fa admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.214483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f582a7-2884-4e22-9959-a44b57f87f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.215406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f582a7-2884-4e22-9959-a44b57f87f4d admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.215406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f582a7-2884-4e22-9959-a44b57f87f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.215687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f582a7-2884-4e22-9959-a44b57f87f4d admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.216269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f582a7-2884-4e22-9959-a44b57f87f4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.218208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f582a7-2884-4e22-9959-a44b57f87f4d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.236973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78d381-d559-4483-888e-033a1e7e125a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.237582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78d381-d559-4483-888e-033a1e7e125a admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.237863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78d381-d559-4483-888e-033a1e7e125a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.238161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78d381-d559-4483-888e-033a1e7e125a admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.238789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78d381-d559-4483-888e-033a1e7e125a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.240681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78d381-d559-4483-888e-033a1e7e125a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.288991 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 620/1252] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:13 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:13.289742 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f582a7-2884-4e22-9959-a44b57f87f4d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.299231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f582a7-2884-4e22-9959-a44b57f87f4d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.303770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7036139-6419-47ea-b0ac-3971df25ed52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.304339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7036139-6419-47ea-b0ac-3971df25ed52 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.304617 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7036139-6419-47ea-b0ac-3971df25ed52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.304886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7036139-6419-47ea-b0ac-3971df25ed52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.305516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7036139-6419-47ea-b0ac-3971df25ed52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.307099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7036139-6419-47ea-b0ac-3971df25ed52 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.317505 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 621/1253] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:13.318309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7036139-6419-47ea-b0ac-3971df25ed52 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.328275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7036139-6419-47ea-b0ac-3971df25ed52 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.332171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 633/1254] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:13 2022] POST /identity/v3/projects => generated 403 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:13.332171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78d381-d559-4483-888e-033a1e7e125a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.336002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c93e2-51f9-4a81-9f27-3d55651b814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.336588 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c93e2-51f9-4a81-9f27-3d55651b814c admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.336868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c93e2-51f9-4a81-9f27-3d55651b814c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.337133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c93e2-51f9-4a81-9f27-3d55651b814c admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.337749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c93e2-51f9-4a81-9f27-3d55651b814c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.339634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c93e2-51f9-4a81-9f27-3d55651b814c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.393770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78d381-d559-4483-888e-033a1e7e125a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.398400 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef3bf2-83c0-465e-beee-6b36cb950e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.399001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef3bf2-83c0-465e-beee-6b36cb950e9a admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.399279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef3bf2-83c0-465e-beee-6b36cb950e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.399552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef3bf2-83c0-465e-beee-6b36cb950e9a admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.400174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef3bf2-83c0-465e-beee-6b36cb950e9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.402106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef3bf2-83c0-465e-beee-6b36cb950e9a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.450686 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 622/1255] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:13 2022] POST /identity/v3/projects => generated 391 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:13.452004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e533b6-7e5d-459f-b4a1-ee6de3c19c29 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.452295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e533b6-7e5d-459f-b4a1-ee6de3c19c29 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.452568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e533b6-7e5d-459f-b4a1-ee6de3c19c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.470677 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c8e533b6-7e5d-459f-b4a1-ee6de3c19c29 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:13.471862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c8e533b6-7e5d-459f-b4a1-ee6de3c19c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:13.506883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 634/1256] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:13 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:13.507708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef3bf2-83c0-465e-beee-6b36cb950e9a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.517090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef3bf2-83c0-465e-beee-6b36cb950e9a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.533176 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 623/1257] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:13.534017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e533b6-7e5d-459f-b4a1-ee6de3c19c29 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.543902 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e533b6-7e5d-459f-b4a1-ee6de3c19c29 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.548440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9ae68-1493-4f38-b725-befb12d665ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.549037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9ae68-1493-4f38-b725-befb12d665ea admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.549318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9ae68-1493-4f38-b725-befb12d665ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.549594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9ae68-1493-4f38-b725-befb12d665ea admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.550261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9ae68-1493-4f38-b725-befb12d665ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.552193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9ae68-1493-4f38-b725-befb12d665ea admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.576123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe314d1-1623-446d-a19d-9feaed57c52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.576725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe314d1-1623-446d-a19d-9feaed57c52b admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.577004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe314d1-1623-446d-a19d-9feaed57c52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.577277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe314d1-1623-446d-a19d-9feaed57c52b admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.577933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe314d1-1623-446d-a19d-9feaed57c52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.579949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe314d1-1623-446d-a19d-9feaed57c52b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.655000 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 624/1258] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:13 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:13.656065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9ae68-1493-4f38-b725-befb12d665ea admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.665688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9ae68-1493-4f38-b725-befb12d665ea admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.670907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65cb4a-f55b-445c-b1be-f88c53f3b9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.671480 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65cb4a-f55b-445c-b1be-f88c53f3b9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.671754 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65cb4a-f55b-445c-b1be-f88c53f3b9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.672022 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc65cb4a-f55b-445c-b1be-f88c53f3b9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.672643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65cb4a-f55b-445c-b1be-f88c53f3b9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.674233 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc65cb4a-f55b-445c-b1be-f88c53f3b9e0 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.678014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:13.685669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 635/1259] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:13 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:13.685669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe314d1-1623-446d-a19d-9feaed57c52b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.690758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 625/1260] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:13.690758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65cb4a-f55b-445c-b1be-f88c53f3b9e0 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.697104 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc65cb4a-f55b-445c-b1be-f88c53f3b9e0 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.698437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe314d1-1623-446d-a19d-9feaed57c52b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.701848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba09707-90be-4dc3-8ca5-b898f11747c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.702457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09707-90be-4dc3-8ca5-b898f11747c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.702731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09707-90be-4dc3-8ca5-b898f11747c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.702996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09707-90be-4dc3-8ca5-b898f11747c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.703177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa979202-52a7-4166-b1a0-6d8dc5e62e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.703610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba09707-90be-4dc3-8ca5-b898f11747c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.703751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa979202-52a7-4166-b1a0-6d8dc5e62e9e admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.704025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa979202-52a7-4166-b1a0-6d8dc5e62e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.704294 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa979202-52a7-4166-b1a0-6d8dc5e62e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.704933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa979202-52a7-4166-b1a0-6d8dc5e62e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.705564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba09707-90be-4dc3-8ca5-b898f11747c9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.706650 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa979202-52a7-4166-b1a0-6d8dc5e62e9e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.716862 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 636/1261] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:13.717631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa979202-52a7-4166-b1a0-6d8dc5e62e9e admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.726674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa979202-52a7-4166-b1a0-6d8dc5e62e9e admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.731871 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9ac2b-d545-48e5-af22-02e2f6c31708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.732448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9ac2b-d545-48e5-af22-02e2f6c31708 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.732730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9ac2b-d545-48e5-af22-02e2f6c31708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.733003 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9ac2b-d545-48e5-af22-02e2f6c31708 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.733789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9ac2b-d545-48e5-af22-02e2f6c31708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.736284 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9ac2b-d545-48e5-af22-02e2f6c31708 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.769408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:13.778247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 626/1262] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:13 2022] POST /identity/v3/users => generated 302 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:13.778247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba09707-90be-4dc3-8ca5-b898f11747c9 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.785476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba09707-90be-4dc3-8ca5-b898f11747c9 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.790019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e60e-a6d2-4761-af00-907922a6f20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.790626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e60e-a6d2-4761-af00-907922a6f20b admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.790904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e60e-a6d2-4761-af00-907922a6f20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.791175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e60e-a6d2-4761-af00-907922a6f20b admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.791808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2e60e-a6d2-4761-af00-907922a6f20b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.792337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 637/1263] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:13 2022] POST /identity/v3/users => generated 296 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:13.793730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9ac2b-d545-48e5-af22-02e2f6c31708 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.794090 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2e60e-a6d2-4761-af00-907922a6f20b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.805635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 627/1264] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:13.806883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e60e-a6d2-4761-af00-907922a6f20b admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.816792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e60e-a6d2-4761-af00-907922a6f20b admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.822496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5b6ec-b11a-45fa-80d0-57562caa31e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.823394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5b6ec-b11a-45fa-80d0-57562caa31e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.823772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5b6ec-b11a-45fa-80d0-57562caa31e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.824141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5b6ec-b11a-45fa-80d0-57562caa31e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.824940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5b6ec-b11a-45fa-80d0-57562caa31e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.826608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5b6ec-b11a-45fa-80d0-57562caa31e4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.837094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 628/1265] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:13.837942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5b6ec-b11a-45fa-80d0-57562caa31e4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.849452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9ac2b-d545-48e5-af22-02e2f6c31708 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.849651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5b6ec-b11a-45fa-80d0-57562caa31e4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.853142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa6d3c-8e56-4225-b470-b98c56d153ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.853753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa6d3c-8e56-4225-b470-b98c56d153ee service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.854033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa6d3c-8e56-4225-b470-b98c56d153ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.854336 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa6d3c-8e56-4225-b470-b98c56d153ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.854476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cd5c9-0f47-4af0-ac8d-92d20179a1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.855061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cd5c9-0f47-4af0-ac8d-92d20179a1fd admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.855339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cd5c9-0f47-4af0-ac8d-92d20179a1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.855605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cd5c9-0f47-4af0-ac8d-92d20179a1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.856872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cd5c9-0f47-4af0-ac8d-92d20179a1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.858474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cd5c9-0f47-4af0-ac8d-92d20179a1fd admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.864141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa6d3c-8e56-4225-b470-b98c56d153ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.865689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa6d3c-8e56-4225-b470-b98c56d153ee service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.871948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 629/1266] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:13.871948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cd5c9-0f47-4af0-ac8d-92d20179a1fd admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.881617 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 638/1267] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:13.882645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa6d3c-8e56-4225-b470-b98c56d153ee service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.889555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cd5c9-0f47-4af0-ac8d-92d20179a1fd admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.892791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b2e3a-5f70-46e6-a02b-593944c5dc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.893131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa6d3c-8e56-4225-b470-b98c56d153ee service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.893858 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b2e3a-5f70-46e6-a02b-593944c5dc9e admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.894379 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b2e3a-5f70-46e6-a02b-593944c5dc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.894857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b2e3a-5f70-46e6-a02b-593944c5dc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.895986 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b2e3a-5f70-46e6-a02b-593944c5dc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.897700 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330555a-0d44-4d03-b12f-3f3e2bc80a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.898378 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330555a-0d44-4d03-b12f-3f3e2bc80a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.898656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330555a-0d44-4d03-b12f-3f3e2bc80a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.898941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330555a-0d44-4d03-b12f-3f3e2bc80a51 admin admin] PATH_INFO: `/v3/projects/17bdff91aa4241abb7a6d880bd58ddd5/users/ccf98cdc6aed4b6b9303acadada2bcfa/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.899268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b2e3a-5f70-46e6-a02b-593944c5dc9e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.914822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 630/1268] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:13.915969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b2e3a-5f70-46e6-a02b-593944c5dc9e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.921655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330555a-0d44-4d03-b12f-3f3e2bc80a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17bdff91aa4241abb7a6d880bd58ddd5, user_id=ccf98cdc6aed4b6b9303acadada2bcfa, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.924528 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330555a-0d44-4d03-b12f-3f3e2bc80a51 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.927028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b2e3a-5f70-46e6-a02b-593944c5dc9e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.931590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ff077-b2e2-4f95-97ce-abac6db16e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.932177 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff077-b2e2-4f95-97ce-abac6db16e5d admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.932449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff077-b2e2-4f95-97ce-abac6db16e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.932721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff077-b2e2-4f95-97ce-abac6db16e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.933338 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ff077-b2e2-4f95-97ce-abac6db16e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.934935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ff077-b2e2-4f95-97ce-abac6db16e5d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.945132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 631/1269] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:13.945899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ff077-b2e2-4f95-97ce-abac6db16e5d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.954883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ff077-b2e2-4f95-97ce-abac6db16e5d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.959409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b360c8-1e8c-4fa0-9ea5-9b2ea4d1f57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.959949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b360c8-1e8c-4fa0-9ea5-9b2ea4d1f57e admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.960222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b360c8-1e8c-4fa0-9ea5-9b2ea4d1f57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.960487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b360c8-1e8c-4fa0-9ea5-9b2ea4d1f57e admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.961086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b360c8-1e8c-4fa0-9ea5-9b2ea4d1f57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.962662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b360c8-1e8c-4fa0-9ea5-9b2ea4d1f57e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:13.972608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 632/1270] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:13.980288 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b360c8-1e8c-4fa0-9ea5-9b2ea4d1f57e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:13.982889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b360c8-1e8c-4fa0-9ea5-9b2ea4d1f57e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:13.987378 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ed84a-97f2-4445-a2ac-184d42511bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:13.987910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ed84a-97f2-4445-a2ac-184d42511bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:13.988180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ed84a-97f2-4445-a2ac-184d42511bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:13.988444 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ed84a-97f2-4445-a2ac-184d42511bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:13.989037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ed84a-97f2-4445-a2ac-184d42511bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:13.990610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ed84a-97f2-4445-a2ac-184d42511bcb admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:14.000970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 633/1271] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:14.001618 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ed84a-97f2-4445-a2ac-184d42511bcb admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:14.012796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ed84a-97f2-4445-a2ac-184d42511bcb admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:14.015257 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 639/1272] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:13 2022] PUT /identity/v3/projects/17bdff91aa4241abb7a6d880bd58ddd5/users/ccf98cdc6aed4b6b9303acadada2bcfa/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:14.020245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330555a-0d44-4d03-b12f-3f3e2bc80a51 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:14.076480 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5d963-8959-45a6-a96e-40020404c96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:14.077134 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5d963-8959-45a6-a96e-40020404c96a admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.077461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5d963-8959-45a6-a96e-40020404c96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.077734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5d963-8959-45a6-a96e-40020404c96a admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.078551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5d963-8959-45a6-a96e-40020404c96a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:14.080137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5d963-8959-45a6-a96e-40020404c96a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:14.094534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 634/1273] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:14.094534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5d963-8959-45a6-a96e-40020404c96a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:14.094773 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330555a-0d44-4d03-b12f-3f3e2bc80a51 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:14.097896 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e6ab7-9559-44e4-855f-606be8c950c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:14.099059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e6ab7-9559-44e4-855f-606be8c950c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.099059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e6ab7-9559-44e4-855f-606be8c950c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.099411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e6ab7-9559-44e4-855f-606be8c950c5 admin admin] PATH_INFO: `/v3/projects/96a3bc9cac034979ab4020109cccd48a/users/d25a54e03d9848bb9747f18254dadd0b/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.102492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5d963-8959-45a6-a96e-40020404c96a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:14.107550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8beebb-233f-4c81-a27f-dbee43d4b1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:14.108615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8beebb-233f-4c81-a27f-dbee43d4b1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.108615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8beebb-233f-4c81-a27f-dbee43d4b1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.108801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8beebb-233f-4c81-a27f-dbee43d4b1e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.109293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8beebb-233f-4c81-a27f-dbee43d4b1e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:14.110898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8beebb-233f-4c81-a27f-dbee43d4b1e5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:14.116507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e6ab7-9559-44e4-855f-606be8c950c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a3bc9cac034979ab4020109cccd48a, user_id=d25a54e03d9848bb9747f18254dadd0b, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:14.119177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e6ab7-9559-44e4-855f-606be8c950c5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:14.122024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 635/1274] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:14.126466 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8beebb-233f-4c81-a27f-dbee43d4b1e5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:14.135666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8beebb-233f-4c81-a27f-dbee43d4b1e5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:14.140759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1602a-3c99-4f85-9bce-d5358a17c585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:14.141321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1602a-3c99-4f85-9bce-d5358a17c585 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.141596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1602a-3c99-4f85-9bce-d5358a17c585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.141864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1602a-3c99-4f85-9bce-d5358a17c585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.142534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1602a-3c99-4f85-9bce-d5358a17c585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:14.144063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1602a-3c99-4f85-9bce-d5358a17c585 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:14.159137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 636/1275] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:14.159137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1602a-3c99-4f85-9bce-d5358a17c585 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:14.164306 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1602a-3c99-4f85-9bce-d5358a17c585 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:14.168910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08368c8b-3a56-440c-a4a4-23eaa1923cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:14.169492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08368c8b-3a56-440c-a4a4-23eaa1923cdc admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.169803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08368c8b-3a56-440c-a4a4-23eaa1923cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.170141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08368c8b-3a56-440c-a4a4-23eaa1923cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.170760 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08368c8b-3a56-440c-a4a4-23eaa1923cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:14.172321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08368c8b-3a56-440c-a4a4-23eaa1923cdc admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:14.182373 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 640/1276] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:14 2022] PUT /identity/v3/projects/96a3bc9cac034979ab4020109cccd48a/users/d25a54e03d9848bb9747f18254dadd0b/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:14.183486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e6ab7-9559-44e4-855f-606be8c950c5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:14.192564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 637/1277] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:14.194479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f052b15-3a29-4537-9113-b0161843e623 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.194887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f052b15-3a29-4537-9113-b0161843e623 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.195245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f052b15-3a29-4537-9113-b0161843e623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.224427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:14.228990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-8f052b15-3a29-4537-9113-b0161843e623 None None] MFA Rules not processed for user `ccf98cdc6aed4b6b9303acadada2bcfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:14.230296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8f052b15-3a29-4537-9113-b0161843e623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:14.270324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e6ab7-9559-44e4-855f-606be8c950c5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:14.271302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:14.277266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31be87-869a-4644-815c-2f126593d486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:14.279258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31be87-869a-4644-815c-2f126593d486 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.279619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31be87-869a-4644-815c-2f126593d486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.279970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31be87-869a-4644-815c-2f126593d486 admin admin] PATH_INFO: `/v3/projects/f7c3d7524e6d4737a780bc3c3978fddc/users/f4a8219a3bf94ad1bf9e0008d58aefa1/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.295792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31be87-869a-4644-815c-2f126593d486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c3d7524e6d4737a780bc3c3978fddc, user_id=f4a8219a3bf94ad1bf9e0008d58aefa1, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:14.298541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31be87-869a-4644-815c-2f126593d486 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:14.304650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 638/1278] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:14.305271 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f052b15-3a29-4537-9113-b0161843e623 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:14.315463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f052b15-3a29-4537-9113-b0161843e623 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:14.320922 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9c75c-dc73-4e6a-93f2-a4b56974c17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:14.321712 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9c75c-dc73-4e6a-93f2-a4b56974c17f admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.322082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9c75c-dc73-4e6a-93f2-a4b56974c17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.322426 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9c75c-dc73-4e6a-93f2-a4b56974c17f admin admin] PATH_INFO: `/v3/projects/0da5eb777b65440caf322db3b33ca98d/users/f72461280ab14c14bb88f88b3867c2f5/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.338529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9c75c-dc73-4e6a-93f2-a4b56974c17f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da5eb777b65440caf322db3b33ca98d, user_id=f72461280ab14c14bb88f88b3867c2f5, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:14.341957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae9c75c-dc73-4e6a-93f2-a4b56974c17f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:14.381084 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 641/1279] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:14 2022] PUT /identity/v3/projects/f7c3d7524e6d4737a780bc3c3978fddc/users/f4a8219a3bf94ad1bf9e0008d58aefa1/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:14.390408 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31be87-869a-4644-815c-2f126593d486 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:14.433734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 639/1280] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:14 2022] PUT /identity/v3/projects/0da5eb777b65440caf322db3b33ca98d/users/f72461280ab14c14bb88f88b3867c2f5/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:14.434792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9c75c-dc73-4e6a-93f2-a4b56974c17f admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:14.446373 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31be87-869a-4644-815c-2f126593d486 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:14.502060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9c75c-dc73-4e6a-93f2-a4b56974c17f admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:14.506908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f28ba7-fdfd-4b4e-a438-7df4c1224155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:14.507580 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f28ba7-fdfd-4b4e-a438-7df4c1224155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.507855 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f28ba7-fdfd-4b4e-a438-7df4c1224155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.508125 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f28ba7-fdfd-4b4e-a438-7df4c1224155 admin admin] PATH_INFO: `/v3/projects/d44801f178bb4a29bfccb298c367a4c2/users/daa27fef364d49ad8240e5f8b353ab2f/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.522157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d026a-70b2-456f-a5a1-9fb046becf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:14.523348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d026a-70b2-456f-a5a1-9fb046becf20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.523678 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d026a-70b2-456f-a5a1-9fb046becf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.523954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d026a-70b2-456f-a5a1-9fb046becf20 admin admin] PATH_INFO: `/v3/projects/ce28abbc9ca441e5828551c3ee01d79c/users/f33275c5521842349cc00e470e60b822/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.535105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f28ba7-fdfd-4b4e-a438-7df4c1224155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d44801f178bb4a29bfccb298c367a4c2, user_id=daa27fef364d49ad8240e5f8b353ab2f, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:14.538758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f28ba7-fdfd-4b4e-a438-7df4c1224155 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:14.548948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d026a-70b2-456f-a5a1-9fb046becf20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce28abbc9ca441e5828551c3ee01d79c, user_id=f33275c5521842349cc00e470e60b822, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:14.551641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d026a-70b2-456f-a5a1-9fb046becf20 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:14.615405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 640/1281] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:14 2022] PUT /identity/v3/projects/d44801f178bb4a29bfccb298c367a4c2/users/daa27fef364d49ad8240e5f8b353ab2f/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:14.616656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578b910-8cc9-4e7c-adfa-b06544250ab6 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.616939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578b910-8cc9-4e7c-adfa-b06544250ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.617205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578b910-8cc9-4e7c-adfa-b06544250ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.636218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-0578b910-8cc9-4e7c-adfa-b06544250ab6 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:14.638015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0578b910-8cc9-4e7c-adfa-b06544250ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:14.654329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 642/1282] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:14 2022] PUT /identity/v3/projects/ce28abbc9ca441e5828551c3ee01d79c/users/f33275c5521842349cc00e470e60b822/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:14.654329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d026a-70b2-456f-a5a1-9fb046becf20 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:14.698626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:14.703888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 641/1283] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:14.705298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93bebd-1de6-4432-8cec-db1ceb8c9b32 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.705587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93bebd-1de6-4432-8cec-db1ceb8c9b32 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.705862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93bebd-1de6-4432-8cec-db1ceb8c9b32 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.707957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 642/1284] 10.222.0.20 () {54 vars in 961 bytes} [Wed Oct 19 02:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:32:14.708762 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a560cc2-915f-438d-85e0-e504b2a40bd0 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.709042 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a560cc2-915f-438d-85e0-e504b2a40bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.709467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a560cc2-915f-438d-85e0-e504b2a40bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.726476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d026a-70b2-456f-a5a1-9fb046becf20 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:14.732988 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe98b2-d2ee-43e4-ac3f-a4a2456bb694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:14.733760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe98b2-d2ee-43e4-ac3f-a4a2456bb694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.734190 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe98b2-d2ee-43e4-ac3f-a4a2456bb694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.734583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe98b2-d2ee-43e4-ac3f-a4a2456bb694 admin admin] PATH_INFO: `/v3/projects/b01bd7d4ec4d49b180bdd62194c92018/users/9f0fad4e9be748c786fe114895d8970c/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.738687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-3a560cc2-915f-438d-85e0-e504b2a40bd0 None None] MFA Rules not processed for user `f4a8219a3bf94ad1bf9e0008d58aefa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:14.739897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3a560cc2-915f-438d-85e0-e504b2a40bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:14.752097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe98b2-d2ee-43e4-ac3f-a4a2456bb694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01bd7d4ec4d49b180bdd62194c92018, user_id=9f0fad4e9be748c786fe114895d8970c, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:14.754882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe98b2-d2ee-43e4-ac3f-a4a2456bb694 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:14.763620 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:14.808439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 643/1285] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:14.809744 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706192a-0843-45b5-8490-4011cea029b8 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.810029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706192a-0843-45b5-8490-4011cea029b8 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.810324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706192a-0843-45b5-8490-4011cea029b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.836870 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2706192a-0843-45b5-8490-4011cea029b8 None None] MFA Rules not processed for user `f72461280ab14c14bb88f88b3867c2f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:14.838067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2706192a-0843-45b5-8490-4011cea029b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:14.883441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 643/1286] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:14 2022] PUT /identity/v3/projects/b01bd7d4ec4d49b180bdd62194c92018/users/9f0fad4e9be748c786fe114895d8970c/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:14.883441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256f404-a09f-4a41-82f2-70bc6f496438 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.883441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256f404-a09f-4a41-82f2-70bc6f496438 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.883441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256f404-a09f-4a41-82f2-70bc6f496438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.901519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-1256f404-a09f-4a41-82f2-70bc6f496438 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:14.903280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1256f404-a09f-4a41-82f2-70bc6f496438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:14.912625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 644/1287] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:14.922170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3fc71-790d-4057-8724-1654fe6783e6 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.922170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3fc71-790d-4057-8724-1654fe6783e6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.922170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3fc71-790d-4057-8724-1654fe6783e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:14.934751 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-4fb3fc71-790d-4057-8724-1654fe6783e6 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:14.935642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4fb3fc71-790d-4057-8724-1654fe6783e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:14.988771 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 644/1288] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:14.988771 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56479d9-dc6b-4b08-98ee-a175e8366ad1 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:14.988771 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56479d9-dc6b-4b08-98ee-a175e8366ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:14.989127 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56479d9-dc6b-4b08-98ee-a175e8366ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.013209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 645/1289] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:15.014794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3fc71-790d-4057-8724-1654fe6783e6 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:15.029618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c56479d9-dc6b-4b08-98ee-a175e8366ad1 None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:15.031002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c56479d9-dc6b-4b08-98ee-a175e8366ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:15.069342 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3fc71-790d-4057-8724-1654fe6783e6 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:15.073783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03c841-23b0-4764-b691-aa0f5b45df7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:15.074420 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c841-23b0-4764-b691-aa0f5b45df7e service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.074732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c841-23b0-4764-b691-aa0f5b45df7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.075004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c841-23b0-4764-b691-aa0f5b45df7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.083898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03c841-23b0-4764-b691-aa0f5b45df7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:15.085320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03c841-23b0-4764-b691-aa0f5b45df7e service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:15.094931 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 646/1290] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:15.095582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03c841-23b0-4764-b691-aa0f5b45df7e service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:15.108214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 645/1291] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Oct 19 02:32:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:15.109957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aff93-6add-4224-beb6-2d5074b89d96 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.109957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aff93-6add-4224-beb6-2d5074b89d96 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.110260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aff93-6add-4224-beb6-2d5074b89d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.112392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03c841-23b0-4764-b691-aa0f5b45df7e service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:15.117716 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228ca73-fac1-455d-ab49-6043a83930cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:15.118546 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228ca73-fac1-455d-ab49-6043a83930cc admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.118986 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228ca73-fac1-455d-ab49-6043a83930cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.119264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228ca73-fac1-455d-ab49-6043a83930cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.119906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228ca73-fac1-455d-ab49-6043a83930cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:15.121870 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228ca73-fac1-455d-ab49-6043a83930cc admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:15.130318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-7e2aff93-6add-4224-beb6-2d5074b89d96 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:15.131522 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7e2aff93-6add-4224-beb6-2d5074b89d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:15.140029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 646/1292] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:15.141290 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e75db-a3d0-4ba4-9c55-5b47969e8cbe None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.141689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e75db-a3d0-4ba4-9c55-5b47969e8cbe None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.142237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e75db-a3d0-4ba4-9c55-5b47969e8cbe None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.143501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 647/1293] 10.222.0.20 () {54 vars in 961 bytes} [Wed Oct 19 02:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:32:15.144569 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e75db-a3d0-4ba4-9c55-5b47969e8cbe None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:15.155449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e75db-a3d0-4ba4-9c55-5b47969e8cbe None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:15.158992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d183f-ecf4-419c-a6ad-c727cae388a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:15.168883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d183f-ecf4-419c-a6ad-c727cae388a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.168883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d183f-ecf4-419c-a6ad-c727cae388a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.168883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926d183f-ecf4-419c-a6ad-c727cae388a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.169471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d183f-ecf4-419c-a6ad-c727cae388a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:15.171085 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926d183f-ecf4-419c-a6ad-c727cae388a9 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:15.180558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 648/1294] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:15.181398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d183f-ecf4-419c-a6ad-c727cae388a9 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:15.191820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926d183f-ecf4-419c-a6ad-c727cae388a9 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:15.197002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49f4c2-4a10-4b21-9c93-6a5ef3aab621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:15.197738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f4c2-4a10-4b21-9c93-6a5ef3aab621 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.198021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f4c2-4a10-4b21-9c93-6a5ef3aab621 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.198372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49f4c2-4a10-4b21-9c93-6a5ef3aab621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.209867 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49f4c2-4a10-4b21-9c93-6a5ef3aab621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:15.211392 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49f4c2-4a10-4b21-9c93-6a5ef3aab621 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:15.224585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 649/1295] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:15.224812 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49f4c2-4a10-4b21-9c93-6a5ef3aab621 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:15.234690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 647/1296] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:15 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:15.236796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6346-091f-4174-9670-3ff654839875 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.237497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6346-091f-4174-9670-3ff654839875 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.237945 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d6346-091f-4174-9670-3ff654839875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.238739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49f4c2-4a10-4b21-9c93-6a5ef3aab621 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:15.253754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:15.265149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ac7d6346-091f-4174-9670-3ff654839875 None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:15.267455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ac7d6346-091f-4174-9670-3ff654839875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:15.330858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcfe8f-84ba-4b6f-a701-6eaab9510e3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:15.332551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcfe8f-84ba-4b6f-a701-6eaab9510e3c service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.337652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcfe8f-84ba-4b6f-a701-6eaab9510e3c service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.337652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcfe8f-84ba-4b6f-a701-6eaab9510e3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.384736 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 648/1297] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Oct 19 02:32:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:15.386279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d6346-091f-4174-9670-3ff654839875 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:15.419355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcfe8f-84ba-4b6f-a701-6eaab9510e3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:15.421289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcfe8f-84ba-4b6f-a701-6eaab9510e3c service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:15.447475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 650/1298] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:15.448860 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcfe8f-84ba-4b6f-a701-6eaab9510e3c service nova] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:15.463394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d6346-091f-4174-9670-3ff654839875 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:15.469253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6dcbe-50da-4881-9976-4d96ee745dcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:15.470989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6dcbe-50da-4881-9976-4d96ee745dcd service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.470989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6dcbe-50da-4881-9976-4d96ee745dcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.470989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6dcbe-50da-4881-9976-4d96ee745dcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.558192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:15.574539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6dcbe-50da-4881-9976-4d96ee745dcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:15.576336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6dcbe-50da-4881-9976-4d96ee745dcd service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:15.583411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcfe8f-84ba-4b6f-a701-6eaab9510e3c service nova] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:15.589179 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb68ce-1db7-4452-ad26-c3c093d8c434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:15.589849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb68ce-1db7-4452-ad26-c3c093d8c434 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.590804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb68ce-1db7-4452-ad26-c3c093d8c434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.590981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 649/1299] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:15.591220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bb68ce-1db7-4452-ad26-c3c093d8c434 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.591914 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb68ce-1db7-4452-ad26-c3c093d8c434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:15.592801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6dcbe-50da-4881-9976-4d96ee745dcd service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:15.593909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bb68ce-1db7-4452-ad26-c3c093d8c434 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:15.609186 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6dcbe-50da-4881-9976-4d96ee745dcd service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:15.617323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea1979-1a15-4612-b9fe-aa948e64ac11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:15.618007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea1979-1a15-4612-b9fe-aa948e64ac11 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.618386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea1979-1a15-4612-b9fe-aa948e64ac11 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.619242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea1979-1a15-4612-b9fe-aa948e64ac11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.705767 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea1979-1a15-4612-b9fe-aa948e64ac11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:15.706124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 651/1300] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:15 2022] POST /identity/v3/users => generated 290 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:15.706745 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea1979-1a15-4612-b9fe-aa948e64ac11 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:15.706994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb68ce-1db7-4452-ad26-c3c093d8c434 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:15.724244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bb68ce-1db7-4452-ad26-c3c093d8c434 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:15.737513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d5d66-ac07-4027-84a4-0f406eef3ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:15.740555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 650/1301] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:15.740692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d5d66-ac07-4027-84a4-0f406eef3ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.740692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d5d66-ac07-4027-84a4-0f406eef3ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.740692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d5d66-ac07-4027-84a4-0f406eef3ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.740692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d5d66-ac07-4027-84a4-0f406eef3ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:15.741094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d5d66-ac07-4027-84a4-0f406eef3ee1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:15.761739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 652/1302] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:32:15.773904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea1979-1a15-4612-b9fe-aa948e64ac11 service nova] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:15.785349 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea1979-1a15-4612-b9fe-aa948e64ac11 service nova] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:15.803233 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f98222-9d26-4af3-8672-b9f392eb4e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:15.803233 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98222-9d26-4af3-8672-b9f392eb4e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.803233 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98222-9d26-4af3-8672-b9f392eb4e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.803233 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f98222-9d26-4af3-8672-b9f392eb4e85 admin admin] PATH_INFO: `/v3/projects/b81bba83bbf14b4fbd61451624b4474c/users/b58555c4830c4889a27152fd47fc60b2/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.851711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f98222-9d26-4af3-8672-b9f392eb4e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81bba83bbf14b4fbd61451624b4474c, user_id=b58555c4830c4889a27152fd47fc60b2, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:15.861277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f98222-9d26-4af3-8672-b9f392eb4e85 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:15.895861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f0e31-e105-4bcb-8acf-10c59058b5ad None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.896212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f0e31-e105-4bcb-8acf-10c59058b5ad None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.896489 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f0e31-e105-4bcb-8acf-10c59058b5ad None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.897961 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 653/1303] 10.222.0.20 () {54 vars in 912 bytes} [Wed Oct 19 02:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:32:15.904428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d374bea-fd4c-4f2b-9032-44ad371a0f2c None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:15.904713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d374bea-fd4c-4f2b-9032-44ad371a0f2c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:15.905239 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d374bea-fd4c-4f2b-9032-44ad371a0f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:15.927411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-9d374bea-fd4c-4f2b-9032-44ad371a0f2c None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:15.928688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9d374bea-fd4c-4f2b-9032-44ad371a0f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:15.941728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 654/1304] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:32:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:15.958388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d374bea-fd4c-4f2b-9032-44ad371a0f2c None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:15.987378 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 651/1305] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:15 2022] PUT /identity/v3/projects/b81bba83bbf14b4fbd61451624b4474c/users/b58555c4830c4889a27152fd47fc60b2/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:15.991116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f98222-9d26-4af3-8672-b9f392eb4e85 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:16.034520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d374bea-fd4c-4f2b-9032-44ad371a0f2c None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:16.109074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:16.136776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0fd88-5c26-467a-839b-fae2dfc79ca5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:16.137655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f98222-9d26-4af3-8672-b9f392eb4e85 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:16.138136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0fd88-5c26-467a-839b-fae2dfc79ca5 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:16.138558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0fd88-5c26-467a-839b-fae2dfc79ca5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:16.139033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0fd88-5c26-467a-839b-fae2dfc79ca5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:16.154394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c90bb5-4983-471b-b684-6f4e655642ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:16.155323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c90bb5-4983-471b-b684-6f4e655642ef admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:16.155865 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c90bb5-4983-471b-b684-6f4e655642ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:16.156337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c90bb5-4983-471b-b684-6f4e655642ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:16.157368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c90bb5-4983-471b-b684-6f4e655642ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:16.159214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c90bb5-4983-471b-b684-6f4e655642ef admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:16.175951 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 652/1306] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:16.184320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace4635-d40a-4a5c-aa14-1874f3bc1982 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:16.184320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace4635-d40a-4a5c-aa14-1874f3bc1982 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:16.184320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace4635-d40a-4a5c-aa14-1874f3bc1982 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:16.185932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 653/1307] 10.222.0.20 () {54 vars in 912 bytes} [Wed Oct 19 02:32:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:32:16.187669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd8b56-82b6-4e92-878e-c2bd7f5fa7ff None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:16.188101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd8b56-82b6-4e92-878e-c2bd7f5fa7ff None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:16.188540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd8b56-82b6-4e92-878e-c2bd7f5fa7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:16.212983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-19cd8b56-82b6-4e92-878e-c2bd7f5fa7ff None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:16.213590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-19cd8b56-82b6-4e92-878e-c2bd7f5fa7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:16.291582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0fd88-5c26-467a-839b-fae2dfc79ca5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:16.293995 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0fd88-5c26-467a-839b-fae2dfc79ca5 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:16.323204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 655/1308] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:32:16.326122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0fd88-5c26-467a-839b-fae2dfc79ca5 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:16.348623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 654/1309] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:32:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:16.364519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:16.367184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd8b56-82b6-4e92-878e-c2bd7f5fa7ff None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:16.382591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0fd88-5c26-467a-839b-fae2dfc79ca5 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:16.387517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd8b56-82b6-4e92-878e-c2bd7f5fa7ff None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:16.393239 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73cdd3-12fd-43c6-8e6a-b0175a2991a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:16.396834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73cdd3-12fd-43c6-8e6a-b0175a2991a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:16.397795 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73cdd3-12fd-43c6-8e6a-b0175a2991a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:16.397998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5579337-1b09-493e-bc23-06025ec1067e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:16.398391 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73cdd3-12fd-43c6-8e6a-b0175a2991a5 admin admin] PATH_INFO: `/v3/projects/b81bba83bbf14b4fbd61451624b4474c/users/b58555c4830c4889a27152fd47fc60b2/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:16.400185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5579337-1b09-493e-bc23-06025ec1067e service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:16.400567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5579337-1b09-493e-bc23-06025ec1067e service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:16.403158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5579337-1b09-493e-bc23-06025ec1067e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:16.404794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73cdd3-12fd-43c6-8e6a-b0175a2991a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81bba83bbf14b4fbd61451624b4474c, user_id=b58555c4830c4889a27152fd47fc60b2, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:16.412386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73cdd3-12fd-43c6-8e6a-b0175a2991a5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:16.440776 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5579337-1b09-493e-bc23-06025ec1067e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:16.442684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5579337-1b09-493e-bc23-06025ec1067e service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:16.471446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 656/1310] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:16 2022] PUT /identity/v3/projects/b81bba83bbf14b4fbd61451624b4474c/users/b58555c4830c4889a27152fd47fc60b2/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:16.475217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18406f-b63a-4524-9973-7d26881ec4e4 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:16.475569 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18406f-b63a-4524-9973-7d26881ec4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:16.475910 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18406f-b63a-4524-9973-7d26881ec4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:16.512766 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 655/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:16.589015 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-9d18406f-b63a-4524-9973-7d26881ec4e4 None None] MFA Rules not processed for user `b58555c4830c4889a27152fd47fc60b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:16.590641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9d18406f-b63a-4524-9973-7d26881ec4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:16.713027 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5579337-1b09-493e-bc23-06025ec1067e service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:16.725443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5579337-1b09-493e-bc23-06025ec1067e service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:16.833775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 657/1312] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:16.833775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18406f-b63a-4524-9973-7d26881ec4e4 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:16.906826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:16.908224 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:16.908510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:16.908781 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] PATH_INFO: `/v3/users/ccf98cdc6aed4b6b9303acadada2bcfa` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:16.916922 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:16.918364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf98cdc6aed4b6b9303acadada2bcfa)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:16.920733 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:16.922459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18406f-b63a-4524-9973-7d26881ec4e4 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:16.937849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33091f8-71ce-45c5-8787-4a83acd22382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:16.939714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33091f8-71ce-45c5-8787-4a83acd22382 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:16.940009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33091f8-71ce-45c5-8787-4a83acd22382 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:16.940341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33091f8-71ce-45c5-8787-4a83acd22382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:16.959112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33091f8-71ce-45c5-8787-4a83acd22382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:16.961169 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33091f8-71ce-45c5-8787-4a83acd22382 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:16.979339 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 658/1313] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:32:17.187041 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:17.220652 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf98cdc6aed4b6b9303acadada2bcfa'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:17.221983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf98cdc6aed4b6b9303acadada2bcfa'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:17.237101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf98cdc6aed4b6b9303acadada2bcfa'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:17.359211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf98cdc6aed4b6b9303acadada2bcfa'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:17.359888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf98cdc6aed4b6b9303acadada2bcfa'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:17.391992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 656/1314] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:16 2022] DELETE /identity/v3/users/ccf98cdc6aed4b6b9303acadada2bcfa => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:17.397117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6df84-2b85-407a-8489-9cc0fd836719 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:17.398102 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6df84-2b85-407a-8489-9cc0fd836719 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:17.398102 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6df84-2b85-407a-8489-9cc0fd836719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:17.424160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-0ac6df84-2b85-407a-8489-9cc0fd836719 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:17.425973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0ac6df84-2b85-407a-8489-9cc0fd836719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:17.504398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 659/1315] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:17.514324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:17.516053 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:17.633877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47edd2-9c28-408a-9a62-7a17742195bb admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:17.639381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cf42a-9886-4385-86c9-55232f86e19b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:17.640890 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf42a-9886-4385-86c9-55232f86e19b service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:17.641343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf42a-9886-4385-86c9-55232f86e19b service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:17.641841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cf42a-9886-4385-86c9-55232f86e19b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:17.674384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cf42a-9886-4385-86c9-55232f86e19b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:17.679163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39cf42a-9886-4385-86c9-55232f86e19b service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:17.705605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 657/1316] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:18.518947 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a36fb-6eca-4ccb-acd3-cce05eb86993 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:18.520096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a36fb-6eca-4ccb-acd3-cce05eb86993 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:18.520530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a36fb-6eca-4ccb-acd3-cce05eb86993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:18.604831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-970a36fb-6eca-4ccb-acd3-cce05eb86993 None None] MFA Rules not processed for user `daa27fef364d49ad8240e5f8b353ab2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:18.606546 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-970a36fb-6eca-4ccb-acd3-cce05eb86993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:18.729734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 660/1317] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:18 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:18.732140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cf42a-9886-4385-86c9-55232f86e19b service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:18.737959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b39cf42a-9886-4385-86c9-55232f86e19b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:18.756100 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39cf42a-9886-4385-86c9-55232f86e19b service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:18.767338 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2963c-fb3e-4e66-9dc9-98f6bfc549e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:18.767970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2963c-fb3e-4e66-9dc9-98f6bfc549e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:18.768249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2963c-fb3e-4e66-9dc9-98f6bfc549e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:18.768522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2963c-fb3e-4e66-9dc9-98f6bfc549e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:18.769171 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2963c-fb3e-4e66-9dc9-98f6bfc549e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:18.773842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2963c-fb3e-4e66-9dc9-98f6bfc549e2 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:18.907184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 658/1318] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:18 2022] POST /identity/v3/projects => generated 405 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:18.908739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a36fb-6eca-4ccb-acd3-cce05eb86993 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:18.934671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a36fb-6eca-4ccb-acd3-cce05eb86993 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:18.983803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:19.071507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2963c-fb3e-4e66-9dc9-98f6bfc549e2 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:19.071507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-39c2963c-fb3e-4e66-9dc9-98f6bfc549e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:19.072756 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd913bc-173e-476c-af5a-2a5f6c5131d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:19.073405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd913bc-173e-476c-af5a-2a5f6c5131d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:19.073662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd913bc-173e-476c-af5a-2a5f6c5131d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:19.073934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd913bc-173e-476c-af5a-2a5f6c5131d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:19.075217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd913bc-173e-476c-af5a-2a5f6c5131d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:19.080800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd913bc-173e-476c-af5a-2a5f6c5131d8 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:19.083959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:19.091302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2963c-fb3e-4e66-9dc9-98f6bfc549e2 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:19.112221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:19.112404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:19.112757 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:19.113098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] PATH_INFO: `/v3/projects/17bdff91aa4241abb7a6d880bd58ddd5` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:19.114334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=17bdff91aa4241abb7a6d880bd58ddd5)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:19.115422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: WARNING py.warnings [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:32:19.115422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: warnings.warn(msg) Oct 19 02:32:19.115422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: Oct 19 02:32:19.116994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:19.187970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 661/1319] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:18 2022] POST /identity/v3/users => generated 309 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:19.189585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd913bc-173e-476c-af5a-2a5f6c5131d8 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:19.212405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd913bc-173e-476c-af5a-2a5f6c5131d8 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:19.221520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239942b1-f3d9-4be5-8001-c828c1648adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:19.222174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239942b1-f3d9-4be5-8001-c828c1648adf admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:19.222474 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239942b1-f3d9-4be5-8001-c828c1648adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:19.222728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239942b1-f3d9-4be5-8001-c828c1648adf admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:19.223381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239942b1-f3d9-4be5-8001-c828c1648adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:19.224998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239942b1-f3d9-4be5-8001-c828c1648adf admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:19.244103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 662/1320] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:19.248215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239942b1-f3d9-4be5-8001-c828c1648adf admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:19.284277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239942b1-f3d9-4be5-8001-c828c1648adf admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:19.308576 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17bdff91aa4241abb7a6d880bd58ddd5', 'request_id': 'req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:19.359356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17bdff91aa4241abb7a6d880bd58ddd5', 'request_id': 'req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:19.427562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52bbac-3779-4da7-851c-e9935d569706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:19.428246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52bbac-3779-4da7-851c-e9935d569706 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:19.428529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52bbac-3779-4da7-851c-e9935d569706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:19.428988 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52bbac-3779-4da7-851c-e9935d569706 admin admin] PATH_INFO: `/v3/projects/01d972bb19fe42c4930ac7d923a6743f/users/9809177a68da41398fa11a71fcf40f74/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:19.447743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52bbac-3779-4da7-851c-e9935d569706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d972bb19fe42c4930ac7d923a6743f, user_id=9809177a68da41398fa11a71fcf40f74, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:19.453235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52bbac-3779-4da7-851c-e9935d569706 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:19.530257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17bdff91aa4241abb7a6d880bd58ddd5', 'request_id': 'req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:19.548873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] The token cache is being invalidate because project 17bdff91aa4241abb7a6d880bd58ddd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:19.549429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:19.551326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-434d5b48-f6bf-49aa-ae30-f59570e8ce2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:19.553748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 659/1321] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:19 2022] DELETE /identity/v3/projects/17bdff91aa4241abb7a6d880bd58ddd5 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:19.570786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0c2f-df8f-4608-9152-e5b1d64ea4de None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:19.571482 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0c2f-df8f-4608-9152-e5b1d64ea4de None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:19.571482 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc0c2f-df8f-4608-9152-e5b1d64ea4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:19.582807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 663/1322] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:19 2022] PUT /identity/v3/projects/01d972bb19fe42c4930ac7d923a6743f/users/9809177a68da41398fa11a71fcf40f74/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:19.584521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52bbac-3779-4da7-851c-e9935d569706 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:19.588335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ad52bbac-3779-4da7-851c-e9935d569706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:19.623804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-68fc0c2f-df8f-4608-9152-e5b1d64ea4de None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:19.625525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-68fc0c2f-df8f-4608-9152-e5b1d64ea4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:19.782867 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52bbac-3779-4da7-851c-e9935d569706 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:19.799413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b64d69-5c07-4fcc-8fb3-6127c878e182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:19.800686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b64d69-5c07-4fcc-8fb3-6127c878e182 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:19.800971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b64d69-5c07-4fcc-8fb3-6127c878e182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:19.802285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 660/1323] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:19.802726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b64d69-5c07-4fcc-8fb3-6127c878e182 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:19.803208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b64d69-5c07-4fcc-8fb3-6127c878e182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:19.810360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae93c8-7a90-4c38-b76d-7085878a8601 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:19.810631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae93c8-7a90-4c38-b76d-7085878a8601 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:19.810903 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae93c8-7a90-4c38-b76d-7085878a8601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:19.826029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b64d69-5c07-4fcc-8fb3-6127c878e182 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:19.849928 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-dcae93c8-7a90-4c38-b76d-7085878a8601 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:19.851171 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-dcae93c8-7a90-4c38-b76d-7085878a8601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:19.861598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 664/1324] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:19.864134 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b64d69-5c07-4fcc-8fb3-6127c878e182 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:19.868098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 661/1325] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:19.872365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae93c8-7a90-4c38-b76d-7085878a8601 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:19.888980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae93c8-7a90-4c38-b76d-7085878a8601 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:19.900112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179b05-9bd3-4b3c-a4d7-ab119a06dd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:19.900537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b64d69-5c07-4fcc-8fb3-6127c878e182 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:19.900741 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179b05-9bd3-4b3c-a4d7-ab119a06dd97 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:19.900999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179b05-9bd3-4b3c-a4d7-ab119a06dd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:19.901269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179b05-9bd3-4b3c-a4d7-ab119a06dd97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:19.901929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8179b05-9bd3-4b3c-a4d7-ab119a06dd97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:19.902255 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8179b05-9bd3-4b3c-a4d7-ab119a06dd97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:19.904395 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8179b05-9bd3-4b3c-a4d7-ab119a06dd97 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:19.912110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3a55d-9bca-4724-8ab8-9b7589d49769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:19.917526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3a55d-9bca-4724-8ab8-9b7589d49769 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:19.917526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3a55d-9bca-4724-8ab8-9b7589d49769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:19.917526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3a55d-9bca-4724-8ab8-9b7589d49769 admin admin] PATH_INFO: `/v3/projects/01d972bb19fe42c4930ac7d923a6743f/users/9809177a68da41398fa11a71fcf40f74/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:19.921525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3a55d-9bca-4724-8ab8-9b7589d49769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d972bb19fe42c4930ac7d923a6743f, user_id=9809177a68da41398fa11a71fcf40f74, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:19.929930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 662/1326] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:32:19.930196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3a55d-9bca-4724-8ab8-9b7589d49769 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:19.934873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179b05-9bd3-4b3c-a4d7-ab119a06dd97 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:19.955809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179b05-9bd3-4b3c-a4d7-ab119a06dd97 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:19.956060 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 665/1327] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:19 2022] PUT /identity/v3/projects/01d972bb19fe42c4930ac7d923a6743f/users/9809177a68da41398fa11a71fcf40f74/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:20.091047 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5f4d0-3f2d-4b24-8e15-9f5b8566c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:20.091754 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5f4d0-3f2d-4b24-8e15-9f5b8566c0d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:20.092101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5f4d0-3f2d-4b24-8e15-9f5b8566c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:20.092380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5f4d0-3f2d-4b24-8e15-9f5b8566c0d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:20.093127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5f4d0-3f2d-4b24-8e15-9f5b8566c0d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:20.097791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5f4d0-3f2d-4b24-8e15-9f5b8566c0d2 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:20.209968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 663/1328] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:19 2022] POST /identity/v3/projects => generated 405 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:20.214678 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3a55d-9bca-4724-8ab8-9b7589d49769 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:20.237429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3a55d-9bca-4724-8ab8-9b7589d49769 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:20.254432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:20.429664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7ee56-6482-4fd6-9b2f-48ed7459535c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:20.430321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7ee56-6482-4fd6-9b2f-48ed7459535c admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:20.430753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7ee56-6482-4fd6-9b2f-48ed7459535c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:20.431025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7ee56-6482-4fd6-9b2f-48ed7459535c admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:20.432011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7ee56-6482-4fd6-9b2f-48ed7459535c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:20.435882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7ee56-6482-4fd6-9b2f-48ed7459535c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:20.558764 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 666/1329] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:20 2022] POST /identity/v3/users => generated 303 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:20.565737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5f4d0-3f2d-4b24-8e15-9f5b8566c0d2 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:20.590968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5f4d0-3f2d-4b24-8e15-9f5b8566c0d2 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:20.611470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:20.621127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ae868-4e90-466d-bdaa-b1c803d8ce81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:20.621127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ae868-4e90-466d-bdaa-b1c803d8ce81 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:20.624404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ae868-4e90-466d-bdaa-b1c803d8ce81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:20.629318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769ae868-4e90-466d-bdaa-b1c803d8ce81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:20.632827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ae868-4e90-466d-bdaa-b1c803d8ce81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:20.645465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769ae868-4e90-466d-bdaa-b1c803d8ce81 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:20.701162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 664/1330] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:20.705184 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7ee56-6482-4fd6-9b2f-48ed7459535c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:20.721705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7ee56-6482-4fd6-9b2f-48ed7459535c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:20.736950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9197c-4278-4642-b726-f4665067346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:20.739732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9197c-4278-4642-b726-f4665067346c admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:20.739732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9197c-4278-4642-b726-f4665067346c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:20.739732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9197c-4278-4642-b726-f4665067346c admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:20.739732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9197c-4278-4642-b726-f4665067346c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:20.740506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9197c-4278-4642-b726-f4665067346c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:20.769710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ae868-4e90-466d-bdaa-b1c803d8ce81 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:20.772189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 667/1331] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:20.776573 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769ae868-4e90-466d-bdaa-b1c803d8ce81 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:20.790836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6f593-b3ad-4b5e-9569-3ec5c7b404fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:20.791518 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6f593-b3ad-4b5e-9569-3ec5c7b404fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:20.791792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6f593-b3ad-4b5e-9569-3ec5c7b404fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:20.792061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6f593-b3ad-4b5e-9569-3ec5c7b404fa admin admin] PATH_INFO: `/v3/projects/8ba05548b4644662aac9c80957cb8139/users/e999aab74cb6424ab85495006161c9e6/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:20.816088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6f593-b3ad-4b5e-9569-3ec5c7b404fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba05548b4644662aac9c80957cb8139, user_id=e999aab74cb6424ab85495006161c9e6, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:20.824245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d6f593-b3ad-4b5e-9569-3ec5c7b404fa admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:20.940587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 665/1332] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:20 2022] PUT /identity/v3/projects/8ba05548b4644662aac9c80957cb8139/users/e999aab74cb6424ab85495006161c9e6/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:20.943189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21df13-d2ca-405a-9d46-5e9b240ea7d2 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:20.943638 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21df13-d2ca-405a-9d46-5e9b240ea7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:20.944058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21df13-d2ca-405a-9d46-5e9b240ea7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:20.985614 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-3c21df13-d2ca-405a-9d46-5e9b240ea7d2 None None] MFA Rules not processed for user `e999aab74cb6424ab85495006161c9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:20.985614 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3c21df13-d2ca-405a-9d46-5e9b240ea7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:21.116842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6f593-b3ad-4b5e-9569-3ec5c7b404fa admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:21.120018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 668/1333] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:20 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:21.272707 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6f593-b3ad-4b5e-9569-3ec5c7b404fa admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:21.301038 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5ece7-044e-4ccf-894e-efd9b8d3e255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:21.303463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5ece7-044e-4ccf-894e-efd9b8d3e255 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:21.305702 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5ece7-044e-4ccf-894e-efd9b8d3e255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:21.306842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5ece7-044e-4ccf-894e-efd9b8d3e255 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:21.311315 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5ece7-044e-4ccf-894e-efd9b8d3e255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:21.317252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5ece7-044e-4ccf-894e-efd9b8d3e255 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:21.474999 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21df13-d2ca-405a-9d46-5e9b240ea7d2 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:21.475845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 666/1334] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:21 2022] POST /identity/v3/projects => generated 405 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:21.494527 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21df13-d2ca-405a-9d46-5e9b240ea7d2 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:21.666626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5ece7-044e-4ccf-894e-efd9b8d3e255 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:21.668556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-01a5ece7-044e-4ccf-894e-efd9b8d3e255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:21.684686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd770-a732-4c1f-b91d-7dff5a50cce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:21.686106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd770-a732-4c1f-b91d-7dff5a50cce4 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:21.686106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd770-a732-4c1f-b91d-7dff5a50cce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:21.686106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd770-a732-4c1f-b91d-7dff5a50cce4 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:21.686741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cd770-a732-4c1f-b91d-7dff5a50cce4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:21.696348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cd770-a732-4c1f-b91d-7dff5a50cce4 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:21.828405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 669/1335] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:21 2022] POST /identity/v3/users => generated 309 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:21.832414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd770-a732-4c1f-b91d-7dff5a50cce4 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:21.866692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd770-a732-4c1f-b91d-7dff5a50cce4 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:21.875142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe52e6-45d0-41b6-8b68-07f2dce1b789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:21.879123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe52e6-45d0-41b6-8b68-07f2dce1b789 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:21.879411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe52e6-45d0-41b6-8b68-07f2dce1b789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:21.879683 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe52e6-45d0-41b6-8b68-07f2dce1b789 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:21.880166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5ece7-044e-4ccf-894e-efd9b8d3e255 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:21.881568 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe52e6-45d0-41b6-8b68-07f2dce1b789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:21.888243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe52e6-45d0-41b6-8b68-07f2dce1b789 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:21.896374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:21.896606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3bc88-e526-4ea7-880b-99f333512509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:21.899133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3bc88-e526-4ea7-880b-99f333512509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:21.899425 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3bc88-e526-4ea7-880b-99f333512509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:21.904525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3bc88-e526-4ea7-880b-99f333512509 admin admin] PATH_INFO: `/v3/users/f4a8219a3bf94ad1bf9e0008d58aefa1` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:21.910913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3bc88-e526-4ea7-880b-99f333512509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4a8219a3bf94ad1bf9e0008d58aefa1)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:21.915398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3bc88-e526-4ea7-880b-99f333512509 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:21.919295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 670/1336] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:21.924494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe52e6-45d0-41b6-8b68-07f2dce1b789 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:21.968852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe52e6-45d0-41b6-8b68-07f2dce1b789 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:21.989372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594f9f8-5f64-41ca-b71d-b72167dc42b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:21.990895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f9f8-5f64-41ca-b71d-b72167dc42b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:21.991736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f9f8-5f64-41ca-b71d-b72167dc42b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:21.992782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594f9f8-5f64-41ca-b71d-b72167dc42b1 admin admin] PATH_INFO: `/v3/projects/2c4aa40bdb31430b9dd1235423795156/users/364510841f42440ea2f9c14ebe231904/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:22.046845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594f9f8-5f64-41ca-b71d-b72167dc42b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4aa40bdb31430b9dd1235423795156, user_id=364510841f42440ea2f9c14ebe231904, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:22.050260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594f9f8-5f64-41ca-b71d-b72167dc42b1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:22.131401 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 671/1337] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:21 2022] PUT /identity/v3/projects/2c4aa40bdb31430b9dd1235423795156/users/364510841f42440ea2f9c14ebe231904/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:22.132958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594f9f8-5f64-41ca-b71d-b72167dc42b1 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:22.206891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-20e3bc88-e526-4ea7-880b-99f333512509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4a8219a3bf94ad1bf9e0008d58aefa1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:22.207738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-20e3bc88-e526-4ea7-880b-99f333512509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4a8219a3bf94ad1bf9e0008d58aefa1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:22.237358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-20e3bc88-e526-4ea7-880b-99f333512509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4a8219a3bf94ad1bf9e0008d58aefa1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:22.341328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-20e3bc88-e526-4ea7-880b-99f333512509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4a8219a3bf94ad1bf9e0008d58aefa1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:22.342287 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-20e3bc88-e526-4ea7-880b-99f333512509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4a8219a3bf94ad1bf9e0008d58aefa1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:22.376203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:22.392908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 667/1338] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:21 2022] DELETE /identity/v3/users/f4a8219a3bf94ad1bf9e0008d58aefa1 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:22.394333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbe747-20d6-4c21-ac3d-73fce8ff47df None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:22.397211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbe747-20d6-4c21-ac3d-73fce8ff47df None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:22.397512 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbe747-20d6-4c21-ac3d-73fce8ff47df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:22.452656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594f9f8-5f64-41ca-b71d-b72167dc42b1 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:22.485977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-d8fbe747-20d6-4c21-ac3d-73fce8ff47df None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:22.489218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d8fbe747-20d6-4c21-ac3d-73fce8ff47df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:22.672607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccf32f-7773-4b54-a472-889d1deb5450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:22.673262 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccf32f-7773-4b54-a472-889d1deb5450 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:22.673531 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccf32f-7773-4b54-a472-889d1deb5450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:22.673803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccf32f-7773-4b54-a472-889d1deb5450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:22.687514 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccf32f-7773-4b54-a472-889d1deb5450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:22.689838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ccf32f-7773-4b54-a472-889d1deb5450 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:22.728904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 672/1339] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 597 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:22.733627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccf32f-7773-4b54-a472-889d1deb5450 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:22.748573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a6ccf32f-7773-4b54-a472-889d1deb5450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:22.796442 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 668/1340] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:22.807458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ccf32f-7773-4b54-a472-889d1deb5450 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:22.823088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2f952-3146-4546-8d94-c250ef63f714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:22.824128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2f952-3146-4546-8d94-c250ef63f714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:22.824128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2f952-3146-4546-8d94-c250ef63f714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:22.824346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2f952-3146-4546-8d94-c250ef63f714 admin admin] PATH_INFO: `/v3/projects/2c4aa40bdb31430b9dd1235423795156/users/364510841f42440ea2f9c14ebe231904/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:22.831966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2f952-3146-4546-8d94-c250ef63f714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c4aa40bdb31430b9dd1235423795156, user_id=364510841f42440ea2f9c14ebe231904, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:22.834703 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2f952-3146-4546-8d94-c250ef63f714 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:22.847437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbe747-20d6-4c21-ac3d-73fce8ff47df None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:22.851015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d8fbe747-20d6-4c21-ac3d-73fce8ff47df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:22.898072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 673/1341] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:22 2022] PUT /identity/v3/projects/2c4aa40bdb31430b9dd1235423795156/users/364510841f42440ea2f9c14ebe231904/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:22.905073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795b341-b746-483b-86bf-723605ad4901 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:22.905073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795b341-b746-483b-86bf-723605ad4901 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:22.905073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795b341-b746-483b-86bf-723605ad4901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:22.994616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-5795b341-b746-483b-86bf-723605ad4901 None None] MFA Rules not processed for user `364510841f42440ea2f9c14ebe231904`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:22.997127 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5795b341-b746-483b-86bf-723605ad4901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:23.027545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbe747-20d6-4c21-ac3d-73fce8ff47df None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:23.167095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:23.189005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875e093-b6c6-4332-97c5-b46164d2c992 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:23.189248 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875e093-b6c6-4332-97c5-b46164d2c992 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:23.189780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875e093-b6c6-4332-97c5-b46164d2c992 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:23.190507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875e093-b6c6-4332-97c5-b46164d2c992 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:23.223480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 674/1342] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:22 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:23.237444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795b341-b746-483b-86bf-723605ad4901 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:23.239110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5795b341-b746-483b-86bf-723605ad4901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:23.336660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875e093-b6c6-4332-97c5-b46164d2c992 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:23.338246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875e093-b6c6-4332-97c5-b46164d2c992 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:23.362330 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795b341-b746-483b-86bf-723605ad4901 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:23.364664 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 669/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:23.366224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcc777-c81c-437c-a012-dbf3c9254f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:23.366849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc777-c81c-437c-a012-dbf3c9254f5e service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:23.367129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc777-c81c-437c-a012-dbf3c9254f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:23.367583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc777-c81c-437c-a012-dbf3c9254f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:23.388656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcc777-c81c-437c-a012-dbf3c9254f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:23.390451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcc777-c81c-437c-a012-dbf3c9254f5e service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:23.412779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 675/1344] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:23 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:32:23.655990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:24.859664 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875e093-b6c6-4332-97c5-b46164d2c992 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:24.880347 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875e093-b6c6-4332-97c5-b46164d2c992 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:24.899772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:24.899772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:24.899772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:24.900057 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] PATH_INFO: `/v3/users/e999aab74cb6424ab85495006161c9e6` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:24.901299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e999aab74cb6424ab85495006161c9e6)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:24.903392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:25.146620 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e999aab74cb6424ab85495006161c9e6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:25.148348 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e999aab74cb6424ab85495006161c9e6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:25.166185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e999aab74cb6424ab85495006161c9e6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:25.249778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e999aab74cb6424ab85495006161c9e6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:25.250704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e999aab74cb6424ab85495006161c9e6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:25.273451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 670/1345] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:24 2022] DELETE /identity/v3/users/e999aab74cb6424ab85495006161c9e6 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:25.278977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d4b9b-3d2e-47c2-a7d8-8e66923762c7 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:25.279571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d4b9b-3d2e-47c2-a7d8-8e66923762c7 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:25.279571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d4b9b-3d2e-47c2-a7d8-8e66923762c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:25.303510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:25.304733 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:25.331010 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-6b8d4b9b-3d2e-47c2-a7d8-8e66923762c7 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:25.332217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6b8d4b9b-3d2e-47c2-a7d8-8e66923762c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:25.456987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfbaba-44d2-42e0-928c-d45495cd3518 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:25.469388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:25.470061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:25.470332 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:25.470645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] PATH_INFO: `/v3/projects/f7c3d7524e6d4737a780bc3c3978fddc` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:25.473253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c3d7524e6d4737a780bc3c3978fddc)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:25.475247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:25.484303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 676/1346] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:25.523498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d4b9b-3d2e-47c2-a7d8-8e66923762c7 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:25.525588 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6b8d4b9b-3d2e-47c2-a7d8-8e66923762c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:25.610534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d4b9b-3d2e-47c2-a7d8-8e66923762c7 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:25.617056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:25.622871 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c3d7524e6d4737a780bc3c3978fddc', 'request_id': 'req-797d85ca-3c69-4112-93ad-1196e32e3e97'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:25.636161 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c3d7524e6d4737a780bc3c3978fddc', 'request_id': 'req-797d85ca-3c69-4112-93ad-1196e32e3e97'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:25.736103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c3d7524e6d4737a780bc3c3978fddc', 'request_id': 'req-797d85ca-3c69-4112-93ad-1196e32e3e97'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:25.753039 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:25.760916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccef8e-6604-4bd9-bc8c-f5e6431c2086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:25.761541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccef8e-6604-4bd9-bc8c-f5e6431c2086 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:25.761818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccef8e-6604-4bd9-bc8c-f5e6431c2086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:25.762695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccef8e-6604-4bd9-bc8c-f5e6431c2086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:25.762885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] The token cache is being invalidate because project f7c3d7524e6d4737a780bc3c3978fddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:25.763272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:25.765700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-797d85ca-3c69-4112-93ad-1196e32e3e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:25.769073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 671/1347] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:25 2022] DELETE /identity/v3/projects/f7c3d7524e6d4737a780bc3c3978fddc => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:25.773150 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa1a96-0b90-4a9d-87c8-89affc0739c2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:25.773408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa1a96-0b90-4a9d-87c8-89affc0739c2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:25.773728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa1a96-0b90-4a9d-87c8-89affc0739c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:25.857978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-89aa1a96-0b90-4a9d-87c8-89affc0739c2 None None] MFA Rules not processed for user `75e5de4fccd243e580a12e2475162451`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:25.859368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-89aa1a96-0b90-4a9d-87c8-89affc0739c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:25.884755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccef8e-6604-4bd9-bc8c-f5e6431c2086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:25.891804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccef8e-6604-4bd9-bc8c-f5e6431c2086 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:25.917216 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 677/1348] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:25.918184 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccef8e-6604-4bd9-bc8c-f5e6431c2086 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:25.945754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccef8e-6604-4bd9-bc8c-f5e6431c2086 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:25.951874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:25.953324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:25.953324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:25.953324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] PATH_INFO: `/v3/users/b58555c4830c4889a27152fd47fc60b2` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:25.954635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b58555c4830c4889a27152fd47fc60b2)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:25.955868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: WARNING py.warnings [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 19 02:32:25.955868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: warnings.warn(msg) Oct 19 02:32:25.955868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: Oct 19 02:32:25.957088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:26.008681 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 672/1349] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:26.034651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa1a96-0b90-4a9d-87c8-89affc0739c2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:26.036685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-89aa1a96-0b90-4a9d-87c8-89affc0739c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:26.181700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa1a96-0b90-4a9d-87c8-89affc0739c2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:26.198032 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebbe51-092b-4bfe-8280-0e7ede4e635a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:26.198695 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebbe51-092b-4bfe-8280-0e7ede4e635a service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:26.198977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebbe51-092b-4bfe-8280-0e7ede4e635a service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:26.200414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebbe51-092b-4bfe-8280-0e7ede4e635a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:26.220019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebbe51-092b-4bfe-8280-0e7ede4e635a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:26.222159 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebbe51-092b-4bfe-8280-0e7ede4e635a service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:26.258572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 673/1350] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:32:26.280657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b58555c4830c4889a27152fd47fc60b2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:26.281391 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b58555c4830c4889a27152fd47fc60b2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:26.409450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b58555c4830c4889a27152fd47fc60b2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:26.410173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b58555c4830c4889a27152fd47fc60b2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:26.434212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b58555c4830c4889a27152fd47fc60b2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:26.486863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 678/1351] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:25 2022] DELETE /identity/v3/users/b58555c4830c4889a27152fd47fc60b2 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:27.212021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709b850-cd2f-46e4-b7bb-1b9931d0a58c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:27.212522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709b850-cd2f-46e4-b7bb-1b9931d0a58c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:27.212606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709b850-cd2f-46e4-b7bb-1b9931d0a58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:27.265973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-a709b850-cd2f-46e4-b7bb-1b9931d0a58c None None] MFA Rules not processed for user `9809177a68da41398fa11a71fcf40f74`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:27.267398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a709b850-cd2f-46e4-b7bb-1b9931d0a58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:27.362632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 674/1352] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:27 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:27.381396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:27.533199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:27.534947 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:27.681955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f342111-231e-49b2-97cb-c3731348692a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:27.689231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:27.690371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:27.691364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:27.692026 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] PATH_INFO: `/v3/projects/8ba05548b4644662aac9c80957cb8139` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:27.694272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba05548b4644662aac9c80957cb8139)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:27.696689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:27.721625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:27.870308 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba05548b4644662aac9c80957cb8139', 'request_id': 'req-d9f645a2-7429-4adc-a377-4761cd38f3e3'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:27.882685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba05548b4644662aac9c80957cb8139', 'request_id': 'req-d9f645a2-7429-4adc-a377-4761cd38f3e3'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:27.921050 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709b850-cd2f-46e4-b7bb-1b9931d0a58c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:27.922533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a709b850-cd2f-46e4-b7bb-1b9931d0a58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:27.950849 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709b850-cd2f-46e4-b7bb-1b9931d0a58c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:27.975941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba05548b4644662aac9c80957cb8139', 'request_id': 'req-d9f645a2-7429-4adc-a377-4761cd38f3e3'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:27.997232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] The token cache is being invalidate because project 8ba05548b4644662aac9c80957cb8139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:27.998889 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.002398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.004325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 679/1353] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:27 2022] DELETE /identity/v3/projects/8ba05548b4644662aac9c80957cb8139 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:28.010491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:28.016473 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:28.078830 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:28.079745 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:28.080122 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:28.081782 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] PATH_INFO: `/v3/projects/b81bba83bbf14b4fbd61451624b4474c` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:28.087645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b81bba83bbf14b4fbd61451624b4474c)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:28.090289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:28.178074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f645a2-7429-4adc-a377-4761cd38f3e3 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:28.186177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:28.187918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:28.187918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:28.187918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] PATH_INFO: `/v3/users/364510841f42440ea2f9c14ebe231904` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:28.188448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=364510841f42440ea2f9c14ebe231904)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:28.190538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:28.268329 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b81bba83bbf14b4fbd61451624b4474c', 'request_id': 'req-e181deb8-4a91-45f4-b747-46269b46e053'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.295421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b81bba83bbf14b4fbd61451624b4474c', 'request_id': 'req-e181deb8-4a91-45f4-b747-46269b46e053'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.376130 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b81bba83bbf14b4fbd61451624b4474c', 'request_id': 'req-e181deb8-4a91-45f4-b747-46269b46e053'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.422425 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] The token cache is being invalidate because project b81bba83bbf14b4fbd61451624b4474c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:28.424826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.429118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e181deb8-4a91-45f4-b747-46269b46e053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.435234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 675/1354] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:27 2022] DELETE /identity/v3/projects/b81bba83bbf14b4fbd61451624b4474c => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:28.440465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebf9b5-5e24-49aa-ade6-859cfaa6442d None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:28.440935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebf9b5-5e24-49aa-ade6-859cfaa6442d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:28.441398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebf9b5-5e24-49aa-ade6-859cfaa6442d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:28.507373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-4bebf9b5-5e24-49aa-ade6-859cfaa6442d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:28.509504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4bebf9b5-5e24-49aa-ade6-859cfaa6442d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:28.529047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364510841f42440ea2f9c14ebe231904'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.531528 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364510841f42440ea2f9c14ebe231904'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.620820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364510841f42440ea2f9c14ebe231904'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.621605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364510841f42440ea2f9c14ebe231904'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.643128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364510841f42440ea2f9c14ebe231904'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:28.680621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 676/1355] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:28.680621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc071fb-7814-46c2-8e05-53fb33f32754 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:28.680621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc071fb-7814-46c2-8e05-53fb33f32754 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:28.680621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc071fb-7814-46c2-8e05-53fb33f32754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:28.698821 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 680/1356] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:28 2022] DELETE /identity/v3/users/364510841f42440ea2f9c14ebe231904 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:28.737597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-6fc071fb-7814-46c2-8e05-53fb33f32754 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:28.737597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6fc071fb-7814-46c2-8e05-53fb33f32754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:28.901247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:28.903067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:28.905510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 677/1357] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:28.909734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc071fb-7814-46c2-8e05-53fb33f32754 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:28.925222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2390c-33ff-48b9-a244-822de0089539 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:28.931843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc071fb-7814-46c2-8e05-53fb33f32754 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:28.939107 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8972bf8-5ef0-4600-b3aa-24a0f21de167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:28.939895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8972bf8-5ef0-4600-b3aa-24a0f21de167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:28.940284 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8972bf8-5ef0-4600-b3aa-24a0f21de167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:28.940661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8972bf8-5ef0-4600-b3aa-24a0f21de167 admin admin] PATH_INFO: `/v3/users/f72461280ab14c14bb88f88b3867c2f5` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:28.942640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8972bf8-5ef0-4600-b3aa-24a0f21de167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72461280ab14c14bb88f88b3867c2f5)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:28.944957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8972bf8-5ef0-4600-b3aa-24a0f21de167 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:28.946111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04d4f0-026b-4540-ad3d-2a61f17357b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:28.947913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04d4f0-026b-4540-ad3d-2a61f17357b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:28.948193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04d4f0-026b-4540-ad3d-2a61f17357b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:28.948929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04d4f0-026b-4540-ad3d-2a61f17357b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:28.953001 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04d4f0-026b-4540-ad3d-2a61f17357b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:28.953325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04d4f0-026b-4540-ad3d-2a61f17357b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:28.958374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04d4f0-026b-4540-ad3d-2a61f17357b9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:28.992343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 678/1358] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:32:28.992676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04d4f0-026b-4540-ad3d-2a61f17357b9 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:29.005569 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04d4f0-026b-4540-ad3d-2a61f17357b9 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:29.012508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b5079-8e11-41a8-8ac7-63e6e4d0705c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:29.013142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b5079-8e11-41a8-8ac7-63e6e4d0705c admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:29.013555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b5079-8e11-41a8-8ac7-63e6e4d0705c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:29.013881 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b5079-8e11-41a8-8ac7-63e6e4d0705c admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:29.014940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b5079-8e11-41a8-8ac7-63e6e4d0705c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:29.017599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b5079-8e11-41a8-8ac7-63e6e4d0705c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:29.164355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:29.202079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 679/1359] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:28 2022] POST /identity/v3/projects => generated 385 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:32:29.206684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b5079-8e11-41a8-8ac7-63e6e4d0705c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:29.323182 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d8972bf8-5ef0-4600-b3aa-24a0f21de167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72461280ab14c14bb88f88b3867c2f5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:29.325747 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d8972bf8-5ef0-4600-b3aa-24a0f21de167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72461280ab14c14bb88f88b3867c2f5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:29.385339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b5079-8e11-41a8-8ac7-63e6e4d0705c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:29.400892 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200103c7-8ac5-4c0d-8d67-2dc7feeb1869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:29.401476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200103c7-8ac5-4c0d-8d67-2dc7feeb1869 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:29.401821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200103c7-8ac5-4c0d-8d67-2dc7feeb1869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:29.402894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200103c7-8ac5-4c0d-8d67-2dc7feeb1869 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:29.405012 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200103c7-8ac5-4c0d-8d67-2dc7feeb1869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:29.409889 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d8972bf8-5ef0-4600-b3aa-24a0f21de167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72461280ab14c14bb88f88b3867c2f5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:29.410856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d8972bf8-5ef0-4600-b3aa-24a0f21de167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72461280ab14c14bb88f88b3867c2f5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:29.412717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200103c7-8ac5-4c0d-8d67-2dc7feeb1869 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:29.422788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d8972bf8-5ef0-4600-b3aa-24a0f21de167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72461280ab14c14bb88f88b3867c2f5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:29.808533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 681/1360] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:28 2022] DELETE /identity/v3/users/f72461280ab14c14bb88f88b3867c2f5 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:29.808533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7305104-c463-4c59-ac75-e1af1948f0da None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:29.808533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7305104-c463-4c59-ac75-e1af1948f0da None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:29.808533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7305104-c463-4c59-ac75-e1af1948f0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:29.812788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 680/1361] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:29 2022] POST /identity/v3/users => generated 293 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:29.812788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200103c7-8ac5-4c0d-8d67-2dc7feeb1869 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:29.812788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-200103c7-8ac5-4c0d-8d67-2dc7feeb1869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:29.814129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-f7305104-c463-4c59-ac75-e1af1948f0da None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:29.814129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f7305104-c463-4c59-ac75-e1af1948f0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:29.841131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200103c7-8ac5-4c0d-8d67-2dc7feeb1869 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:29.846407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:29.846407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 682/1362] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:29.846407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7305104-c463-4c59-ac75-e1af1948f0da None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:29.846407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f7305104-c463-4c59-ac75-e1af1948f0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:29.848983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a7357-893d-46a8-9a7f-f6cba7b5ffb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:29.848983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a7357-893d-46a8-9a7f-f6cba7b5ffb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:29.848983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a7357-893d-46a8-9a7f-f6cba7b5ffb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:29.848983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a7357-893d-46a8-9a7f-f6cba7b5ffb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:29.849614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a7357-893d-46a8-9a7f-f6cba7b5ffb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:29.849614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a7357-893d-46a8-9a7f-f6cba7b5ffb5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:29.849614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 681/1363] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:29.849614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a7357-893d-46a8-9a7f-f6cba7b5ffb5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:29.849614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a7357-893d-46a8-9a7f-f6cba7b5ffb5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:29.850194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002b8dc-c133-4529-9212-602bfd7715d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:29.850194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002b8dc-c133-4529-9212-602bfd7715d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:29.850194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002b8dc-c133-4529-9212-602bfd7715d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:29.850194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002b8dc-c133-4529-9212-602bfd7715d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:29.850985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002b8dc-c133-4529-9212-602bfd7715d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:29.850985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002b8dc-c133-4529-9212-602bfd7715d4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:29.862466 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 682/1364] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:29.863744 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7305104-c463-4c59-ac75-e1af1948f0da None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:29.870479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002b8dc-c133-4529-9212-602bfd7715d4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:29.873648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e4379-76a7-4787-b127-cf33d2deb5eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:29.874309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e4379-76a7-4787-b127-cf33d2deb5eb service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:29.874850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e4379-76a7-4787-b127-cf33d2deb5eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:29.875280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e4379-76a7-4787-b127-cf33d2deb5eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:29.890717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002b8dc-c133-4529-9212-602bfd7715d4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:29.897199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e4379-76a7-4787-b127-cf33d2deb5eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:29.899624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e4379-76a7-4787-b127-cf33d2deb5eb service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:29.915447 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cdd1b-fc45-47b3-bf42-c10c5cb4d5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:29.915960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cdd1b-fc45-47b3-bf42-c10c5cb4d5d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:29.916251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cdd1b-fc45-47b3-bf42-c10c5cb4d5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:29.916507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cdd1b-fc45-47b3-bf42-c10c5cb4d5d0 admin admin] PATH_INFO: `/v3/projects/e192184bd42842c7918cb539ff38224f/users/7ac0f9e28e6d4183a99e7080779a3056/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:29.922311 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 683/1365] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:32:29.937807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975cdd1b-fc45-47b3-bf42-c10c5cb4d5d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e192184bd42842c7918cb539ff38224f, user_id=7ac0f9e28e6d4183a99e7080779a3056, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:29.941409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975cdd1b-fc45-47b3-bf42-c10c5cb4d5d0 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:30.137080 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 683/1366] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:29 2022] PUT /identity/v3/projects/e192184bd42842c7918cb539ff38224f/users/7ac0f9e28e6d4183a99e7080779a3056/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:30.149025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bf0d2-e114-4418-bc92-54068a5b5f63 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:30.149865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bf0d2-e114-4418-bc92-54068a5b5f63 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:30.157078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bf0d2-e114-4418-bc92-54068a5b5f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:30.184096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5b98-d8a0-4793-9e6d-4b1b3b23d3b9 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:30.184609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5b98-d8a0-4793-9e6d-4b1b3b23d3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:30.185021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d5b98-d8a0-4793-9e6d-4b1b3b23d3b9 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:30.186610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 684/1367] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:32:30.225860 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8b4bf0d2-e114-4418-bc92-54068a5b5f63 None None] MFA Rules not processed for user `7ac0f9e28e6d4183a99e7080779a3056`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:30.227318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8b4bf0d2-e114-4418-bc92-54068a5b5f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:30.235377 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb0f19-52e2-45cf-8a82-a5fcd3c23cca None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:30.235857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb0f19-52e2-45cf-8a82-a5fcd3c23cca None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:30.236269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb0f19-52e2-45cf-8a82-a5fcd3c23cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:30.289451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-e7eb0f19-52e2-45cf-8a82-a5fcd3c23cca None None] MFA Rules not processed for user `9243ebe24ae84157b74696c29bda3279`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:30.293564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e7eb0f19-52e2-45cf-8a82-a5fcd3c23cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:30.477572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 684/1368] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:30.483580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bf0d2-e114-4418-bc92-54068a5b5f63 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:30.539975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 685/1369] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:32:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:30.605628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb0f19-52e2-45cf-8a82-a5fcd3c23cca None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:30.607174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e7eb0f19-52e2-45cf-8a82-a5fcd3c23cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:30.648752 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb0f19-52e2-45cf-8a82-a5fcd3c23cca None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:30.673315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bf0d2-e114-4418-bc92-54068a5b5f63 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:30.676292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8514238-7ae4-43ac-b3c8-9d11f46a2e07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:30.677094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8514238-7ae4-43ac-b3c8-9d11f46a2e07 service placement] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:30.678184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8514238-7ae4-43ac-b3c8-9d11f46a2e07 service placement] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:30.678899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8514238-7ae4-43ac-b3c8-9d11f46a2e07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:30.688235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68922d-a6cb-4a03-894b-41913a48dd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:30.689673 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68922d-a6cb-4a03-894b-41913a48dd58 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:30.690520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68922d-a6cb-4a03-894b-41913a48dd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:30.693016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68922d-a6cb-4a03-894b-41913a48dd58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:30.694975 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68922d-a6cb-4a03-894b-41913a48dd58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:30.697077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68922d-a6cb-4a03-894b-41913a48dd58 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:30.709059 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8514238-7ae4-43ac-b3c8-9d11f46a2e07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:30.711124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8514238-7ae4-43ac-b3c8-9d11f46a2e07 service placement] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:30.734154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 686/1370] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:32:30.739802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8514238-7ae4-43ac-b3c8-9d11f46a2e07 service placement] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:30.745414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e8514238-7ae4-43ac-b3c8-9d11f46a2e07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:30.785981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8514238-7ae4-43ac-b3c8-9d11f46a2e07 service placement] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:30.797824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:30.798890 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:30.799311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:30.799860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] PATH_INFO: `/v3/projects/2c4aa40bdb31430b9dd1235423795156` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:30.805228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c4aa40bdb31430b9dd1235423795156)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:30.807439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:30.852370 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 685/1371] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:30 2022] POST /identity/v3/projects => generated 383 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:30.856790 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68922d-a6cb-4a03-894b-41913a48dd58 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:31.000587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c4aa40bdb31430b9dd1235423795156', 'request_id': 'req-e8223786-4e60-4aff-b74e-2730dcb1104c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:31.013401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c4aa40bdb31430b9dd1235423795156', 'request_id': 'req-e8223786-4e60-4aff-b74e-2730dcb1104c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:31.056831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68922d-a6cb-4a03-894b-41913a48dd58 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:31.164291 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c4aa40bdb31430b9dd1235423795156', 'request_id': 'req-e8223786-4e60-4aff-b74e-2730dcb1104c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:31.210770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] The token cache is being invalidate because project 2c4aa40bdb31430b9dd1235423795156 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:31.211084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:31.214195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8223786-4e60-4aff-b74e-2730dcb1104c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:31.221054 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 687/1372] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:30 2022] DELETE /identity/v3/projects/2c4aa40bdb31430b9dd1235423795156 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:31.225022 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67f142-25e7-49e0-af16-8f32fbfb6b83 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:31.225317 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67f142-25e7-49e0-af16-8f32fbfb6b83 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:31.225596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67f142-25e7-49e0-af16-8f32fbfb6b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:31.276837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-de67f142-25e7-49e0-af16-8f32fbfb6b83 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:31.283110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b821fd0-4d82-4271-9632-63dea0b656e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:31.283794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821fd0-4d82-4271-9632-63dea0b656e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:31.284081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821fd0-4d82-4271-9632-63dea0b656e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:31.284556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b821fd0-4d82-4271-9632-63dea0b656e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:31.285224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b821fd0-4d82-4271-9632-63dea0b656e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:31.289668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-de67f142-25e7-49e0-af16-8f32fbfb6b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:31.296743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b821fd0-4d82-4271-9632-63dea0b656e5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:31.309605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 688/1373] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:31.317260 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d972a9-d4e9-49ca-97e0-f5b5c66bd34e None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:31.317772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d972a9-d4e9-49ca-97e0-f5b5c66bd34e None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:31.318217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d972a9-d4e9-49ca-97e0-f5b5c66bd34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:31.396466 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-b1d972a9-d4e9-49ca-97e0-f5b5c66bd34e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:31.398011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b1d972a9-d4e9-49ca-97e0-f5b5c66bd34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:31.417767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 686/1374] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:30 2022] POST /identity/v3/users => generated 298 bytes in 562 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:31.422463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:31.423500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b821fd0-4d82-4271-9632-63dea0b656e5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:31.427672 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5b821fd0-4d82-4271-9632-63dea0b656e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:31.436217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 689/1375] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:31.443147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d972a9-d4e9-49ca-97e0-f5b5c66bd34e None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:31.467478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d972a9-d4e9-49ca-97e0-f5b5c66bd34e None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:31.470801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b821fd0-4d82-4271-9632-63dea0b656e5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:31.484650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193283f-a391-4ce0-8a61-2e0638861c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:31.485336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193283f-a391-4ce0-8a61-2e0638861c17 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:31.486787 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193283f-a391-4ce0-8a61-2e0638861c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:31.486787 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193283f-a391-4ce0-8a61-2e0638861c17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:31.486787 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193283f-a391-4ce0-8a61-2e0638861c17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:31.487132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193283f-a391-4ce0-8a61-2e0638861c17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:31.494381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193283f-a391-4ce0-8a61-2e0638861c17 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:31.498944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c648a7-2222-4a2f-90a3-9ab01a5eb782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:31.499592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c648a7-2222-4a2f-90a3-9ab01a5eb782 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:31.500130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c648a7-2222-4a2f-90a3-9ab01a5eb782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:31.500397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c648a7-2222-4a2f-90a3-9ab01a5eb782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:31.501171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c648a7-2222-4a2f-90a3-9ab01a5eb782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:31.508376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c648a7-2222-4a2f-90a3-9ab01a5eb782 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:31.534270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 687/1376] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:31.536984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 690/1377] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:32:31.547842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c648a7-2222-4a2f-90a3-9ab01a5eb782 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:31.548030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193283f-a391-4ce0-8a61-2e0638861c17 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:31.570261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193283f-a391-4ce0-8a61-2e0638861c17 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:31.572604 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c648a7-2222-4a2f-90a3-9ab01a5eb782 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:31.580365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3c564-f583-46d3-9ca4-ea6b545dd536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:31.581278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3c564-f583-46d3-9ca4-ea6b545dd536 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:31.587120 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3c564-f583-46d3-9ca4-ea6b545dd536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:31.587538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3c564-f583-46d3-9ca4-ea6b545dd536 admin admin] PATH_INFO: `/v3/projects/c80f580aa50c4107b309f34df0851823/users/1add5f2118ae48d28297f18910c38228/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:31.590254 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3a7ff-4081-40cc-a455-d95aa73a009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:31.591415 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3a7ff-4081-40cc-a455-d95aa73a009d admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:31.593224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3a7ff-4081-40cc-a455-d95aa73a009d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:31.593658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3a7ff-4081-40cc-a455-d95aa73a009d admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:31.594487 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3a7ff-4081-40cc-a455-d95aa73a009d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:31.607358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3a7ff-4081-40cc-a455-d95aa73a009d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:31.629940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3c564-f583-46d3-9ca4-ea6b545dd536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80f580aa50c4107b309f34df0851823, user_id=1add5f2118ae48d28297f18910c38228, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:31.636307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3c564-f583-46d3-9ca4-ea6b545dd536 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:31.757004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 688/1378] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:31 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:31.761266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3a7ff-4081-40cc-a455-d95aa73a009d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:31.784976 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:31.786295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 691/1379] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:31 2022] PUT /identity/v3/projects/c80f580aa50c4107b309f34df0851823/users/1add5f2118ae48d28297f18910c38228/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:31.790822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3c564-f583-46d3-9ca4-ea6b545dd536 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:31.954367 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3c564-f583-46d3-9ca4-ea6b545dd536 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:31.969678 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18149a2-9dab-455b-9943-325eb0a36c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:31.970813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18149a2-9dab-455b-9943-325eb0a36c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:31.971195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18149a2-9dab-455b-9943-325eb0a36c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:31.971354 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18149a2-9dab-455b-9943-325eb0a36c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:31.972274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18149a2-9dab-455b-9943-325eb0a36c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:31.978883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18149a2-9dab-455b-9943-325eb0a36c94 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:32.008358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 692/1380] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:32.009442 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18149a2-9dab-455b-9943-325eb0a36c94 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:32.031047 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18149a2-9dab-455b-9943-325eb0a36c94 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:32.044195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e139a2f-11ce-495d-8223-f05fac9264e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:32.044892 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139a2f-11ce-495d-8223-f05fac9264e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:32.045173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139a2f-11ce-495d-8223-f05fac9264e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:32.045447 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139a2f-11ce-495d-8223-f05fac9264e7 admin admin] PATH_INFO: `/v3/projects/c80f580aa50c4107b309f34df0851823/users/1add5f2118ae48d28297f18910c38228/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:32.049688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e139a2f-11ce-495d-8223-f05fac9264e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80f580aa50c4107b309f34df0851823, user_id=1add5f2118ae48d28297f18910c38228, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:32.052371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e139a2f-11ce-495d-8223-f05fac9264e7 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:32.091874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 693/1381] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:32 2022] PUT /identity/v3/projects/c80f580aa50c4107b309f34df0851823/users/1add5f2118ae48d28297f18910c38228/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:32.104694 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c4e3b-f3ea-49b7-872b-c8b358141bab None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:32.105085 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c4e3b-f3ea-49b7-872b-c8b358141bab None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:32.105409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c4e3b-f3ea-49b7-872b-c8b358141bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:32.169633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3a7ff-4081-40cc-a455-d95aa73a009d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:32.183253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-bb5c4e3b-f3ea-49b7-872b-c8b358141bab None None] MFA Rules not processed for user `1add5f2118ae48d28297f18910c38228`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:32.188967 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-bb5c4e3b-f3ea-49b7-872b-c8b358141bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:32.335322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4ebf6-eddd-455b-8eaf-5a747f080cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:32.336226 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4ebf6-eddd-455b-8eaf-5a747f080cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:32.336570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4ebf6-eddd-455b-8eaf-5a747f080cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:32.336907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4ebf6-eddd-455b-8eaf-5a747f080cc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:32.337904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4ebf6-eddd-455b-8eaf-5a747f080cc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:32.339952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4ebf6-eddd-455b-8eaf-5a747f080cc1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:32.359714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:32.404838 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 694/1382] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:32.410908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c4e3b-f3ea-49b7-872b-c8b358141bab None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:32.412742 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-bb5c4e3b-f3ea-49b7-872b-c8b358141bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:32.437246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c4e3b-f3ea-49b7-872b-c8b358141bab None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:32.437442 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:32.438018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:32.438338 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:32.438616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] PATH_INFO: `/v3/projects/0da5eb777b65440caf322db3b33ca98d` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:32.439982 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0da5eb777b65440caf322db3b33ca98d)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:32.442113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:32.472968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 689/1383] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:31 2022] POST /identity/v3/users => generated 283 bytes in 712 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:32:32.474117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4ebf6-eddd-455b-8eaf-5a747f080cc1 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:32.488055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-02c4ebf6-eddd-455b-8eaf-5a747f080cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:32.603162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0da5eb777b65440caf322db3b33ca98d', 'request_id': 'req-ae404e54-5d9c-4de2-8655-03832f55f4f0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:32.618748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0da5eb777b65440caf322db3b33ca98d', 'request_id': 'req-ae404e54-5d9c-4de2-8655-03832f55f4f0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:32.656969 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4ebf6-eddd-455b-8eaf-5a747f080cc1 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:32.735835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0da5eb777b65440caf322db3b33ca98d', 'request_id': 'req-ae404e54-5d9c-4de2-8655-03832f55f4f0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:32.754571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] The token cache is being invalidate because project 0da5eb777b65440caf322db3b33ca98d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:32.754964 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:32.756259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:32.763136 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 695/1384] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:32 2022] DELETE /identity/v3/projects/0da5eb777b65440caf322db3b33ca98d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:32.764092 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:32.768750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7b441-9771-43b9-8795-44e587dccc81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:32.768885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:32.771997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7b441-9771-43b9-8795-44e587dccc81 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:32.772451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7b441-9771-43b9-8795-44e587dccc81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:32.772892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7b441-9771-43b9-8795-44e587dccc81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:32.775460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a9b7b441-9771-43b9-8795-44e587dccc81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:32.913358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7b441-9771-43b9-8795-44e587dccc81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:32.915780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7b441-9771-43b9-8795-44e587dccc81 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:32.929809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae404e54-5d9c-4de2-8655-03832f55f4f0 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:32.937036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 690/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:32.937036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728897f0-9cf1-4d87-a8ee-da6c2cdb0a93 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:32.937036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728897f0-9cf1-4d87-a8ee-da6c2cdb0a93 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:32.937036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728897f0-9cf1-4d87-a8ee-da6c2cdb0a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:32.954649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b865a2-b23e-407a-aa30-66d42d3004c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:32.955272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b865a2-b23e-407a-aa30-66d42d3004c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:32.955686 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b865a2-b23e-407a-aa30-66d42d3004c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:32.955967 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b865a2-b23e-407a-aa30-66d42d3004c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:32.956705 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b865a2-b23e-407a-aa30-66d42d3004c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:32.958818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b865a2-b23e-407a-aa30-66d42d3004c8 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:32.964632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-728897f0-9cf1-4d87-a8ee-da6c2cdb0a93 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:32.966514 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-728897f0-9cf1-4d87-a8ee-da6c2cdb0a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:32.972252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 696/1386] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:32.977886 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 691/1387] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:32.978077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b865a2-b23e-407a-aa30-66d42d3004c8 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:32.985841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09d48e-da63-45b8-b3e9-b3324fcf9b7b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:32.986347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09d48e-da63-45b8-b3e9-b3324fcf9b7b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:32.986760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09d48e-da63-45b8-b3e9-b3324fcf9b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.006075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b865a2-b23e-407a-aa30-66d42d3004c8 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:33.031123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-7e09d48e-da63-45b8-b3e9-b3324fcf9b7b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:33.031343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc740bbf-0776-410d-a5c4-45197ceb5d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:33.031439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7e09d48e-da63-45b8-b3e9-b3324fcf9b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:33.031540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc740bbf-0776-410d-a5c4-45197ceb5d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:33.031540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc740bbf-0776-410d-a5c4-45197ceb5d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:33.036545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc740bbf-0776-410d-a5c4-45197ceb5d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.038181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc740bbf-0776-410d-a5c4-45197ceb5d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:33.041699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc740bbf-0776-410d-a5c4-45197ceb5d15 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:33.051346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 692/1388] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:33.061947 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09d48e-da63-45b8-b3e9-b3324fcf9b7b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:33.072199 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 697/1389] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:33.073167 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc740bbf-0776-410d-a5c4-45197ceb5d15 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:33.087044 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-bc740bbf-0776-410d-a5c4-45197ceb5d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:33.088213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09d48e-da63-45b8-b3e9-b3324fcf9b7b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:33.100782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba41a0-b1c9-4e6f-b44a-7dbe5016d7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:33.100782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba41a0-b1c9-4e6f-b44a-7dbe5016d7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:33.100782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba41a0-b1c9-4e6f-b44a-7dbe5016d7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:33.100782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba41a0-b1c9-4e6f-b44a-7dbe5016d7a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.101235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba41a0-b1c9-4e6f-b44a-7dbe5016d7a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:33.101235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba41a0-b1c9-4e6f-b44a-7dbe5016d7a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:33.105461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba41a0-b1c9-4e6f-b44a-7dbe5016d7a3 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:33.125154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc740bbf-0776-410d-a5c4-45197ceb5d15 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:33.136804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bfba1-15be-4cf6-94a2-f224408c789e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:33.141640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bfba1-15be-4cf6-94a2-f224408c789e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:33.141640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bfba1-15be-4cf6-94a2-f224408c789e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:33.141640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bfba1-15be-4cf6-94a2-f224408c789e admin admin] PATH_INFO: `/v3/users/7ac0f9e28e6d4183a99e7080779a3056` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.141640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bfba1-15be-4cf6-94a2-f224408c789e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ac0f9e28e6d4183a99e7080779a3056)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:33.142389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bfba1-15be-4cf6-94a2-f224408c789e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:33.146440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 693/1390] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:33.147658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba41a0-b1c9-4e6f-b44a-7dbe5016d7a3 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:33.173340 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba41a0-b1c9-4e6f-b44a-7dbe5016d7a3 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:33.186541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75723a6a-b8a9-4b31-8564-fc9b562b8aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:33.187367 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75723a6a-b8a9-4b31-8564-fc9b562b8aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:33.187755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75723a6a-b8a9-4b31-8564-fc9b562b8aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:33.188131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75723a6a-b8a9-4b31-8564-fc9b562b8aa9 admin admin] PATH_INFO: `/v3/projects/0a4ca42d6f1e4e4e92b778ae0a907d5f/users/169232b5f6a84d919d0f25fcbab330aa/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.193430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:33.232812 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75723a6a-b8a9-4b31-8564-fc9b562b8aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4ca42d6f1e4e4e92b778ae0a907d5f, user_id=169232b5f6a84d919d0f25fcbab330aa, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:33.235739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75723a6a-b8a9-4b31-8564-fc9b562b8aa9 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:33.335339 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 694/1391] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:33 2022] PUT /identity/v3/projects/0a4ca42d6f1e4e4e92b778ae0a907d5f/users/169232b5f6a84d919d0f25fcbab330aa/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:33.336347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75723a6a-b8a9-4b31-8564-fc9b562b8aa9 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:33.419624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75723a6a-b8a9-4b31-8564-fc9b562b8aa9 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:33.420358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-827bfba1-15be-4cf6-94a2-f224408c789e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ac0f9e28e6d4183a99e7080779a3056'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:33.422328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-827bfba1-15be-4cf6-94a2-f224408c789e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ac0f9e28e6d4183a99e7080779a3056'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:33.425832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecc15f-538c-489b-8952-d0bc3156049f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:33.426658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecc15f-538c-489b-8952-d0bc3156049f admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:33.427288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecc15f-538c-489b-8952-d0bc3156049f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:33.427649 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecc15f-538c-489b-8952-d0bc3156049f admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.428853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecc15f-538c-489b-8952-d0bc3156049f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:33.431273 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecc15f-538c-489b-8952-d0bc3156049f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:33.438329 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-827bfba1-15be-4cf6-94a2-f224408c789e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ac0f9e28e6d4183a99e7080779a3056'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:33.530926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-827bfba1-15be-4cf6-94a2-f224408c789e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ac0f9e28e6d4183a99e7080779a3056'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:33.532033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-827bfba1-15be-4cf6-94a2-f224408c789e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ac0f9e28e6d4183a99e7080779a3056'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:33.540285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 695/1392] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:33 2022] POST /identity/v3/projects => generated 381 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:33.542631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dc8ef-d60a-44be-94be-6488baadbefe None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:33.542631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dc8ef-d60a-44be-94be-6488baadbefe None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:33.542631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075dc8ef-d60a-44be-94be-6488baadbefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.585162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 698/1393] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:33 2022] DELETE /identity/v3/users/7ac0f9e28e6d4183a99e7080779a3056 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:33.587888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49709f75-d130-4bc5-9c78-fe315562b22d None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:33.588935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49709f75-d130-4bc5-9c78-fe315562b22d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:33.589537 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49709f75-d130-4bc5-9c78-fe315562b22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.592247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-075dc8ef-d60a-44be-94be-6488baadbefe None None] MFA Rules not processed for user `169232b5f6a84d919d0f25fcbab330aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:33.593505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-075dc8ef-d60a-44be-94be-6488baadbefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:33.627258 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-49709f75-d130-4bc5-9c78-fe315562b22d None None] MFA Rules not processed for user `f33275c5521842349cc00e470e60b822`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:33.630163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-49709f75-d130-4bc5-9c78-fe315562b22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:33.634325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:33.732881 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 696/1394] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:33.734186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dc8ef-d60a-44be-94be-6488baadbefe None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:33.738670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-075dc8ef-d60a-44be-94be-6488baadbefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:33.808243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 699/1395] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:33.809020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dcad7-e75a-434e-93ee-f3edaba3dcd1 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:33.809724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dcad7-e75a-434e-93ee-f3edaba3dcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:33.809724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dcad7-e75a-434e-93ee-f3edaba3dcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.849226 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-890dcad7-e75a-434e-93ee-f3edaba3dcd1 None None] MFA Rules not processed for user `9f0fad4e9be748c786fe114895d8970c`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:33.850598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-890dcad7-e75a-434e-93ee-f3edaba3dcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:33.888889 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075dc8ef-d60a-44be-94be-6488baadbefe None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:33.894452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2172b-b4c1-4683-8db4-f306da6b6d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:33.895913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2172b-b4c1-4683-8db4-f306da6b6d10 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:33.896374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2172b-b4c1-4683-8db4-f306da6b6d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:33.896812 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2172b-b4c1-4683-8db4-f306da6b6d10 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.901575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2172b-b4c1-4683-8db4-f306da6b6d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:33.907005 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2172b-b4c1-4683-8db4-f306da6b6d10 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:33.959737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 700/1396] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:33 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:33.961035 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6baa9-fbb3-43ab-90f5-6c0480dc1223 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:33.961321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6baa9-fbb3-43ab-90f5-6c0480dc1223 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:33.961597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6baa9-fbb3-43ab-90f5-6c0480dc1223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.982978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ffa6baa9-fbb3-43ab-90f5-6c0480dc1223 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:33.984203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ffa6baa9-fbb3-43ab-90f5-6c0480dc1223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:33.996764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 701/1397] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:33.996764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2e9cb-7452-4891-8372-afd1aee44555 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:33.996764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2e9cb-7452-4891-8372-afd1aee44555 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:33.996764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2e9cb-7452-4891-8372-afd1aee44555 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:33.998763 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 702/1398] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:32:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:32:34.000744 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2e9cb-7452-4891-8372-afd1aee44555 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:34.004029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e1b2e9cb-7452-4891-8372-afd1aee44555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.010586 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 697/1399] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:33 2022] POST /identity/v3/users => generated 291 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:34.012135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a8a1b-ce8c-4a13-a37e-fadfe6597c7c None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:34.012464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a8a1b-ce8c-4a13-a37e-fadfe6597c7c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:34.012742 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a8a1b-ce8c-4a13-a37e-fadfe6597c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:34.027673 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2e9cb-7452-4891-8372-afd1aee44555 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:34.034946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-3f6a8a1b-ce8c-4a13-a37e-fadfe6597c7c None None] MFA Rules not processed for user `f33275c5521842349cc00e470e60b822`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:34.035096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c30ef2-a02e-4a06-af7b-2bcc7031b2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:34.035601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c30ef2-a02e-4a06-af7b-2bcc7031b2de admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:34.035889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c30ef2-a02e-4a06-af7b-2bcc7031b2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:34.036448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c30ef2-a02e-4a06-af7b-2bcc7031b2de admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:34.037105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c30ef2-a02e-4a06-af7b-2bcc7031b2de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:34.037379 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3f6a8a1b-ce8c-4a13-a37e-fadfe6597c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.039230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c30ef2-a02e-4a06-af7b-2bcc7031b2de admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:34.045567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 698/1400] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:34.050980 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88997d0a-85b8-41cb-8ef5-69738bb7e70d None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:34.051333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88997d0a-85b8-41cb-8ef5-69738bb7e70d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:34.051680 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88997d0a-85b8-41cb-8ef5-69738bb7e70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:34.062507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:34.085052 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-88997d0a-85b8-41cb-8ef5-69738bb7e70d None None] MFA Rules not processed for user `d25a54e03d9848bb9747f18254dadd0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:34.086445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-88997d0a-85b8-41cb-8ef5-69738bb7e70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.160327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 699/1401] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:34.161026 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88997d0a-85b8-41cb-8ef5-69738bb7e70d None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:34.162453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-88997d0a-85b8-41cb-8ef5-69738bb7e70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.175333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 703/1402] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:33 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:34.177595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0e57a-f59f-4df6-a77f-7f2280f08eec None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:34.177934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0e57a-f59f-4df6-a77f-7f2280f08eec None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:34.178301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0e57a-f59f-4df6-a77f-7f2280f08eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:34.198417 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88997d0a-85b8-41cb-8ef5-69738bb7e70d None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:34.217268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-f4c0e57a-f59f-4df6-a77f-7f2280f08eec None None] MFA Rules not processed for user `75e5de4fccd243e580a12e2475162451`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:34.217268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f4c0e57a-f59f-4df6-a77f-7f2280f08eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.272621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:34.273562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:34.273917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:34.274141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] PATH_INFO: `/v3/users/daa27fef364d49ad8240e5f8b353ab2f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:34.275833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] RBAC: Authorizing `identity:delete_user(user_id=daa27fef364d49ad8240e5f8b353ab2f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:34.277909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:34.278197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 704/1403] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:34.280507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c734a-ea9d-4851-900b-7f0ad68a6cc1 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:34.280791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c734a-ea9d-4851-900b-7f0ad68a6cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:34.281117 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c734a-ea9d-4851-900b-7f0ad68a6cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:34.311567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-6b5c734a-ea9d-4851-900b-7f0ad68a6cc1 None None] MFA Rules not processed for user `9f0fad4e9be748c786fe114895d8970c`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:34.312867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6b5c734a-ea9d-4851-900b-7f0ad68a6cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.411391 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 705/1404] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:34.415844 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6ae63-ec58-4cb2-8469-6bd00aa846b2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:34.416178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6ae63-ec58-4cb2-8469-6bd00aa846b2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:34.416455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6ae63-ec58-4cb2-8469-6bd00aa846b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:34.451312 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-3bf6ae63-ec58-4cb2-8469-6bd00aa846b2 None None] MFA Rules not processed for user `9243ebe24ae84157b74696c29bda3279`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:34.452776 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3bf6ae63-ec58-4cb2-8469-6bd00aa846b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.523894 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daa27fef364d49ad8240e5f8b353ab2f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:34.524780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daa27fef364d49ad8240e5f8b353ab2f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:34.547170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 706/1405] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:32:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:34.548130 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6ae63-ec58-4cb2-8469-6bd00aa846b2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:34.549928 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3bf6ae63-ec58-4cb2-8469-6bd00aa846b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.570857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6ae63-ec58-4cb2-8469-6bd00aa846b2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:34.579284 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c48fdf-3b81-4e26-adec-8ea1d7bb76f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:34.581323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48fdf-3b81-4e26-adec-8ea1d7bb76f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:34.581677 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48fdf-3b81-4e26-adec-8ea1d7bb76f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:34.581962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48fdf-3b81-4e26-adec-8ea1d7bb76f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:34.585854 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c48fdf-3b81-4e26-adec-8ea1d7bb76f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:34.587739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c48fdf-3b81-4e26-adec-8ea1d7bb76f4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:34.599768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 707/1406] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:34.600740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c48fdf-3b81-4e26-adec-8ea1d7bb76f4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:34.602911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e1c48fdf-3b81-4e26-adec-8ea1d7bb76f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.632270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daa27fef364d49ad8240e5f8b353ab2f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:34.634876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daa27fef364d49ad8240e5f8b353ab2f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:34.648619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daa27fef364d49ad8240e5f8b353ab2f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:34.679437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 700/1407] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:34 2022] DELETE /identity/v3/users/daa27fef364d49ad8240e5f8b353ab2f => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:34.680509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:34.683121 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.707021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c48fdf-3b81-4e26-adec-8ea1d7bb76f4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:34.811129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394785df-5d5c-42c3-8169-0a5e26cc478d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:34.818392 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f725c19-e249-43a1-9da5-7942b4f67b5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:34.819326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f725c19-e249-43a1-9da5-7942b4f67b5e service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:34.819326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f725c19-e249-43a1-9da5-7942b4f67b5e service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:34.819819 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f725c19-e249-43a1-9da5-7942b4f67b5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:34.824624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:34.826542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3f725c19-e249-43a1-9da5-7942b4f67b5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.866364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e600ea0-4195-465f-8cf2-4189642efb05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:34.867445 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e600ea0-4195-465f-8cf2-4189642efb05 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:34.869314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e600ea0-4195-465f-8cf2-4189642efb05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:34.869740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e600ea0-4195-465f-8cf2-4189642efb05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:34.874324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5e600ea0-4195-465f-8cf2-4189642efb05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:34.878684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:34.917645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f725c19-e249-43a1-9da5-7942b4f67b5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:34.921119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f725c19-e249-43a1-9da5-7942b4f67b5e service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:34.950270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 701/1408] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:34.951547 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f725c19-e249-43a1-9da5-7942b4f67b5e service nova] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:34.968468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3f725c19-e249-43a1-9da5-7942b4f67b5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:35.012757 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e600ea0-4195-465f-8cf2-4189642efb05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:35.014198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e600ea0-4195-465f-8cf2-4189642efb05 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:35.025580 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 708/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:35.037769 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f725c19-e249-43a1-9da5-7942b4f67b5e service nova] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:35.039343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e600ea0-4195-465f-8cf2-4189642efb05 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:35.044371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5e600ea0-4195-465f-8cf2-4189642efb05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:35.053361 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d727edc-75ac-4510-b0a7-a881b9c5a6fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:35.054077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d727edc-75ac-4510-b0a7-a881b9c5a6fa service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:35.054614 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d727edc-75ac-4510-b0a7-a881b9c5a6fa service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:35.054890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d727edc-75ac-4510-b0a7-a881b9c5a6fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:35.060541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8d727edc-75ac-4510-b0a7-a881b9c5a6fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:35.070387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e600ea0-4195-465f-8cf2-4189642efb05 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:35.082469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e2b8e-9c44-4862-ab40-5727b60eacd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:35.083099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2b8e-9c44-4862-ab40-5727b60eacd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:35.083389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2b8e-9c44-4862-ab40-5727b60eacd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:35.083669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e2b8e-9c44-4862-ab40-5727b60eacd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:35.085030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e2b8e-9c44-4862-ab40-5727b60eacd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:35.087008 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e2b8e-9c44-4862-ab40-5727b60eacd9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:35.211510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 709/1410] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:35 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:35.211510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e2b8e-9c44-4862-ab40-5727b60eacd9 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:35.212206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-205e2b8e-9c44-4862-ab40-5727b60eacd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:35.259378 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d727edc-75ac-4510-b0a7-a881b9c5a6fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:35.260479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d727edc-75ac-4510-b0a7-a881b9c5a6fa service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:35.300129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 702/1411] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:35.301767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d727edc-75ac-4510-b0a7-a881b9c5a6fa service nova] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:35.342460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d727edc-75ac-4510-b0a7-a881b9c5a6fa service nova] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:35.359746 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c5022-db07-49d3-8e37-b28482f6aef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:35.360899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5022-db07-49d3-8e37-b28482f6aef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:35.361501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5022-db07-49d3-8e37-b28482f6aef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:35.369082 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c5022-db07-49d3-8e37-b28482f6aef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:35.369996 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c5022-db07-49d3-8e37-b28482f6aef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:35.373074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c5022-db07-49d3-8e37-b28482f6aef5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:35.376651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e2b8e-9c44-4862-ab40-5727b60eacd9 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:35.392538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424b9e9-a2d1-438e-9c36-a2cae077e1ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:35.393721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b9e9-a2d1-438e-9c36-a2cae077e1ad service placement] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:35.393721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b9e9-a2d1-438e-9c36-a2cae077e1ad service placement] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:35.393721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424b9e9-a2d1-438e-9c36-a2cae077e1ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:35.415496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424b9e9-a2d1-438e-9c36-a2cae077e1ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:35.415717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 703/1412] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:35.417310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c5022-db07-49d3-8e37-b28482f6aef5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:35.420581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424b9e9-a2d1-438e-9c36-a2cae077e1ad service placement] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:35.424065 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-671c5022-db07-49d3-8e37-b28482f6aef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:35.454517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 710/1413] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:35.455599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424b9e9-a2d1-438e-9c36-a2cae077e1ad service placement] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:35.477866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424b9e9-a2d1-438e-9c36-a2cae077e1ad service placement] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:35.482221 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c5022-db07-49d3-8e37-b28482f6aef5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:35.490219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc5d13-dc5d-4f6c-8b15-9cb18d23938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:35.492536 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc5d13-dc5d-4f6c-8b15-9cb18d23938c admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:35.493937 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc5d13-dc5d-4f6c-8b15-9cb18d23938c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:35.493937 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc5d13-dc5d-4f6c-8b15-9cb18d23938c admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:35.494311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc5d13-dc5d-4f6c-8b15-9cb18d23938c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:35.502822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc5d13-dc5d-4f6c-8b15-9cb18d23938c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:35.510318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:35.514442 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:35.515796 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:35.518693 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] PATH_INFO: `/v3/projects/d44801f178bb4a29bfccb298c367a4c2` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:35.525329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d44801f178bb4a29bfccb298c367a4c2)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:35.529103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:35.530232 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 711/1414] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:35 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:35.531244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc5d13-dc5d-4f6c-8b15-9cb18d23938c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:35.558146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc5d13-dc5d-4f6c-8b15-9cb18d23938c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:35.571943 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789daf87-eb9c-4a30-b750-b2ab78570606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:35.572643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789daf87-eb9c-4a30-b750-b2ab78570606 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:35.572987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789daf87-eb9c-4a30-b750-b2ab78570606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:35.573251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789daf87-eb9c-4a30-b750-b2ab78570606 admin admin] PATH_INFO: `/v3/projects/7fc712d2b62445a8bc9a90516c970dee/users/5269b87577834012ad5d2336f115f78c/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:35.608995 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789daf87-eb9c-4a30-b750-b2ab78570606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc712d2b62445a8bc9a90516c970dee, user_id=5269b87577834012ad5d2336f115f78c, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:35.611758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789daf87-eb9c-4a30-b750-b2ab78570606 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:35.681013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:35.714342 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 712/1415] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:35 2022] PUT /identity/v3/projects/7fc712d2b62445a8bc9a90516c970dee/users/5269b87577834012ad5d2336f115f78c/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:35.717236 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789daf87-eb9c-4a30-b750-b2ab78570606 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:35.794071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd44801f178bb4a29bfccb298c367a4c2', 'request_id': 'req-74340161-3c5c-4ce0-bd63-0da7bd4da691'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:35.805355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd44801f178bb4a29bfccb298c367a4c2', 'request_id': 'req-74340161-3c5c-4ce0-bd63-0da7bd4da691'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:35.895880 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd44801f178bb4a29bfccb298c367a4c2', 'request_id': 'req-74340161-3c5c-4ce0-bd63-0da7bd4da691'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:35.898550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789daf87-eb9c-4a30-b750-b2ab78570606 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:35.915466 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] The token cache is being invalidate because project d44801f178bb4a29bfccb298c367a4c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:35.916275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:35.917209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-74340161-3c5c-4ce0-bd63-0da7bd4da691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:35.923133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 704/1416] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:35 2022] DELETE /identity/v3/projects/d44801f178bb4a29bfccb298c367a4c2 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:35.926502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41083e8-82ca-464c-8bb6-b2082507cd31 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:35.927016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41083e8-82ca-464c-8bb6-b2082507cd31 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:35.928138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41083e8-82ca-464c-8bb6-b2082507cd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:36.017482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-d41083e8-82ca-464c-8bb6-b2082507cd31 None None] MFA Rules not processed for user `5269b87577834012ad5d2336f115f78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:36.019011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d41083e8-82ca-464c-8bb6-b2082507cd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:36.084460 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005cd1a-b2bd-45c7-aad1-b9981f933ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:36.086081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005cd1a-b2bd-45c7-aad1-b9981f933ffd admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:36.086585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005cd1a-b2bd-45c7-aad1-b9981f933ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:36.087164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005cd1a-b2bd-45c7-aad1-b9981f933ffd admin admin] PATH_INFO: `/v3/projects/d7ef6b1951f64104acf8414fc1f31cc5/users/922fec5eef034f9f93569d4c59efac60/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:36.116296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 705/1417] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:36.117684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41083e8-82ca-464c-8bb6-b2082507cd31 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:36.123459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d41083e8-82ca-464c-8bb6-b2082507cd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:36.144293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41083e8-82ca-464c-8bb6-b2082507cd31 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:36.152380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:36.153098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:36.154074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:36.154324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005cd1a-b2bd-45c7-aad1-b9981f933ffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ef6b1951f64104acf8414fc1f31cc5, user_id=922fec5eef034f9f93569d4c59efac60, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:36.154428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] PATH_INFO: `/v3/users/9809177a68da41398fa11a71fcf40f74` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:36.155482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9809177a68da41398fa11a71fcf40f74)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:36.157637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:36.167728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005cd1a-b2bd-45c7-aad1-b9981f933ffd admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:36.374723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 713/1418] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:35 2022] PUT /identity/v3/projects/d7ef6b1951f64104acf8414fc1f31cc5/users/922fec5eef034f9f93569d4c59efac60/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:36.379684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42078678-e348-4a90-bc28-01a3417d46d7 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:36.380573 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42078678-e348-4a90-bc28-01a3417d46d7 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:36.381860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42078678-e348-4a90-bc28-01a3417d46d7 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:36.388337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 714/1419] 10.222.0.20 () {54 vars in 912 bytes} [Wed Oct 19 02:32:36 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:32:36.392606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42078678-e348-4a90-bc28-01a3417d46d7 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:36.394141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-42078678-e348-4a90-bc28-01a3417d46d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:36.433680 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9809177a68da41398fa11a71fcf40f74'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:36.435469 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9809177a68da41398fa11a71fcf40f74'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:36.515347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9809177a68da41398fa11a71fcf40f74'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:36.515653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9809177a68da41398fa11a71fcf40f74'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:36.524635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9809177a68da41398fa11a71fcf40f74'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:36.574217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 706/1420] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:36 2022] DELETE /identity/v3/users/9809177a68da41398fa11a71fcf40f74 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:36.579315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:36.582810 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:36.671434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:36.672702 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42078678-e348-4a90-bc28-01a3417d46d7 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:36.719932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1ea0f-8e49-4a81-a745-7cea0af58530 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:36.726380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:36.727276 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:36.728185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:36.728592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] PATH_INFO: `/v3/projects/e192184bd42842c7918cb539ff38224f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:36.729967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e192184bd42842c7918cb539ff38224f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:36.732925 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:36.859628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a56960-b38c-4f0b-8bf2-255f5f71ec6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:36.860616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a56960-b38c-4f0b-8bf2-255f5f71ec6e admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:36.861089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a56960-b38c-4f0b-8bf2-255f5f71ec6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:36.861568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a56960-b38c-4f0b-8bf2-255f5f71ec6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:36.862803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a56960-b38c-4f0b-8bf2-255f5f71ec6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:36.867294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a56960-b38c-4f0b-8bf2-255f5f71ec6e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:36.881312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e192184bd42842c7918cb539ff38224f', 'request_id': 'req-685e0b0e-1fb4-424c-a82b-9355f8c16069'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:36.888931 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e192184bd42842c7918cb539ff38224f', 'request_id': 'req-685e0b0e-1fb4-424c-a82b-9355f8c16069'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:36.997841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e192184bd42842c7918cb539ff38224f', 'request_id': 'req-685e0b0e-1fb4-424c-a82b-9355f8c16069'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.013207 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] The token cache is being invalidate because project e192184bd42842c7918cb539ff38224f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:37.013551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.014800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.022109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 707/1421] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:36 2022] DELETE /identity/v3/projects/e192184bd42842c7918cb539ff38224f => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:37.022109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:37.024933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:37.029957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 715/1422] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:36 2022] POST /identity/v3/projects => generated 379 bytes in 640 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:37.032867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434bd0f-b0ae-4eeb-83ee-07708eaae2f1 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:37.033310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434bd0f-b0ae-4eeb-83ee-07708eaae2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:37.033641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434bd0f-b0ae-4eeb-83ee-07708eaae2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:37.084772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-d434bd0f-b0ae-4eeb-83ee-07708eaae2f1 None None] MFA Rules not processed for user `b5ff8583997843258e315413abac172f`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:37.086313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d434bd0f-b0ae-4eeb-83ee-07708eaae2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:37.102912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:37.226845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e0b0e-1fb4-424c-a82b-9355f8c16069 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:37.229517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 716/1423] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:32:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:37.230138 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b0e63-0df7-41b2-bcbd-fc4b35326f09 None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:37.230443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b0e63-0df7-41b2-bcbd-fc4b35326f09 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:37.230878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b0e63-0df7-41b2-bcbd-fc4b35326f09 None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:37.232517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 717/1424] 10.222.0.20 () {54 vars in 912 bytes} [Wed Oct 19 02:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:32:37.233242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b0e63-0df7-41b2-bcbd-fc4b35326f09 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:37.235127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-303b0e63-0df7-41b2-bcbd-fc4b35326f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:37.363534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc1791-9361-40c5-b2e4-374b6f8481fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:37.364356 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc1791-9361-40c5-b2e4-374b6f8481fa admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:37.364652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc1791-9361-40c5-b2e4-374b6f8481fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:37.364929 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc1791-9361-40c5-b2e4-374b6f8481fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:37.365595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc1791-9361-40c5-b2e4-374b6f8481fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:37.370300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc1791-9361-40c5-b2e4-374b6f8481fa admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:37.373921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b0e63-0df7-41b2-bcbd-fc4b35326f09 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:37.381695 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcf6c2-7ab1-440c-812a-034bd7a09b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:37.382759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcf6c2-7ab1-440c-812a-034bd7a09b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:37.383184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcf6c2-7ab1-440c-812a-034bd7a09b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:37.383603 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcf6c2-7ab1-440c-812a-034bd7a09b70 admin admin] PATH_INFO: `/v3/users/1add5f2118ae48d28297f18910c38228` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:37.384908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcf6c2-7ab1-440c-812a-034bd7a09b70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1add5f2118ae48d28297f18910c38228)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:37.385755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 708/1425] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:37.387036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc1791-9361-40c5-b2e4-374b6f8481fa admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:37.387630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcf6c2-7ab1-440c-812a-034bd7a09b70 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:37.392911 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b4fc1791-9361-40c5-b2e4-374b6f8481fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:37.422379 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc1791-9361-40c5-b2e4-374b6f8481fa admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:37.428413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ee5de-0ffe-4655-816a-7de8ddc1415a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:37.429121 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ee5de-0ffe-4655-816a-7de8ddc1415a admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:37.431050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ee5de-0ffe-4655-816a-7de8ddc1415a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:37.431050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ee5de-0ffe-4655-816a-7de8ddc1415a admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:37.431050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ee5de-0ffe-4655-816a-7de8ddc1415a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:37.433025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ee5de-0ffe-4655-816a-7de8ddc1415a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:37.520569 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 709/1426] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:37 2022] POST /identity/v3/users => generated 296 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:37.521718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ee5de-0ffe-4655-816a-7de8ddc1415a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:37.523850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-069ee5de-0ffe-4655-816a-7de8ddc1415a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:37.539721 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ee5de-0ffe-4655-816a-7de8ddc1415a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:37.545922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:37.546934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:37.547320 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:37.547705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] PATH_INFO: `/v3/projects/01d972bb19fe42c4930ac7d923a6743f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:37.549058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d972bb19fe42c4930ac7d923a6743f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:37.551204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:37.591974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1dbcf6c2-7ab1-440c-812a-034bd7a09b70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1add5f2118ae48d28297f18910c38228'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.593088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1dbcf6c2-7ab1-440c-812a-034bd7a09b70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1add5f2118ae48d28297f18910c38228'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.614096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1dbcf6c2-7ab1-440c-812a-034bd7a09b70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1add5f2118ae48d28297f18910c38228'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.721657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d972bb19fe42c4930ac7d923a6743f', 'request_id': 'req-dda17330-be04-41f4-b3de-f504b2d0efe8'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.729592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d972bb19fe42c4930ac7d923a6743f', 'request_id': 'req-dda17330-be04-41f4-b3de-f504b2d0efe8'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.738772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1dbcf6c2-7ab1-440c-812a-034bd7a09b70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1add5f2118ae48d28297f18910c38228'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.739836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1dbcf6c2-7ab1-440c-812a-034bd7a09b70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1add5f2118ae48d28297f18910c38228'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.770853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 718/1427] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:37 2022] DELETE /identity/v3/users/1add5f2118ae48d28297f18910c38228 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:37.776069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad4e7b-a165-4c08-afd8-c43343728ab2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:37.776430 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad4e7b-a165-4c08-afd8-c43343728ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:37.776842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad4e7b-a165-4c08-afd8-c43343728ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:37.801302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-64ad4e7b-a165-4c08-afd8-c43343728ab2 None None] MFA Rules not processed for user `b5ff8583997843258e315413abac172f`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:37.802845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-64ad4e7b-a165-4c08-afd8-c43343728ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:37.827324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d972bb19fe42c4930ac7d923a6743f', 'request_id': 'req-dda17330-be04-41f4-b3de-f504b2d0efe8'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.845969 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] The token cache is being invalidate because project 01d972bb19fe42c4930ac7d923a6743f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:37.846445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.848481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:37.853726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 710/1428] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:37 2022] DELETE /identity/v3/projects/01d972bb19fe42c4930ac7d923a6743f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:37.853726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:37.853990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:37.913144 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:37.946304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 719/1429] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:32:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:37.948170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad4e7b-a165-4c08-afd8-c43343728ab2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:37.955978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-64ad4e7b-a165-4c08-afd8-c43343728ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:38.018082 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda17330-be04-41f4-b3de-f504b2d0efe8 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:38.029447 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b9e11-ecb4-42d1-8d89-2aa7b5006cea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:38.030899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b9e11-ecb4-42d1-8d89-2aa7b5006cea service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:38.031532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b9e11-ecb4-42d1-8d89-2aa7b5006cea service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:38.032203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b9e11-ecb4-42d1-8d89-2aa7b5006cea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:38.037572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6d4b9e11-ecb4-42d1-8d89-2aa7b5006cea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:38.065578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b9e11-ecb4-42d1-8d89-2aa7b5006cea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:38.068745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b9e11-ecb4-42d1-8d89-2aa7b5006cea service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:38.090349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 711/1430] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:38.092221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad4e7b-a165-4c08-afd8-c43343728ab2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:38.093089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b9e11-ecb4-42d1-8d89-2aa7b5006cea service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:38.095637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6d4b9e11-ecb4-42d1-8d89-2aa7b5006cea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:38.120562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b4a4f-38e8-42cf-b261-c2b8fc4eed29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:38.120562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b4a4f-38e8-42cf-b261-c2b8fc4eed29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:38.120562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b4a4f-38e8-42cf-b261-c2b8fc4eed29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:38.120562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b4a4f-38e8-42cf-b261-c2b8fc4eed29 admin admin] PATH_INFO: `/v3/projects/d7ef6b1951f64104acf8414fc1f31cc5/users/922fec5eef034f9f93569d4c59efac60/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:38.123061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b4a4f-38e8-42cf-b261-c2b8fc4eed29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ef6b1951f64104acf8414fc1f31cc5, user_id=922fec5eef034f9f93569d4c59efac60, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:38.127336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b4a4f-38e8-42cf-b261-c2b8fc4eed29 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:38.141197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b9e11-ecb4-42d1-8d89-2aa7b5006cea service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:38.178114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 720/1431] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:37 2022] PUT /identity/v3/projects/d7ef6b1951f64104acf8414fc1f31cc5/users/922fec5eef034f9f93569d4c59efac60/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:38.180705 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a260c-6775-4c61-a771-746653f97696 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:38.180997 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a260c-6775-4c61-a771-746653f97696 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:38.181274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a260c-6775-4c61-a771-746653f97696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:38.233637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-992a260c-6775-4c61-a771-746653f97696 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:38.238335 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-992a260c-6775-4c61-a771-746653f97696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:38.260158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a025bc-1426-453e-9a2c-913bba82d323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:38.261012 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a025bc-1426-453e-9a2c-913bba82d323 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:38.261445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a025bc-1426-453e-9a2c-913bba82d323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:38.261865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a025bc-1426-453e-9a2c-913bba82d323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:38.263778 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a025bc-1426-453e-9a2c-913bba82d323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:38.265743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a025bc-1426-453e-9a2c-913bba82d323 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:38.293480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 712/1432] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:38.293480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a025bc-1426-453e-9a2c-913bba82d323 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:38.298341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-61a025bc-1426-453e-9a2c-913bba82d323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:38.360069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 721/1433] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:38.360069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7362c6-3884-4eb0-8ec4-a46a534bacff None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:38.360069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7362c6-3884-4eb0-8ec4-a46a534bacff None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:38.360069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7362c6-3884-4eb0-8ec4-a46a534bacff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:38.394460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a025bc-1426-453e-9a2c-913bba82d323 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:38.398334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9b7362c6-3884-4eb0-8ec4-a46a534bacff None None] MFA Rules not processed for user `922fec5eef034f9f93569d4c59efac60`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:38.398519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a70ce-8f87-464d-b061-6d3f19bca834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:38.399136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a70ce-8f87-464d-b061-6d3f19bca834 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:38.399414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a70ce-8f87-464d-b061-6d3f19bca834 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:38.399684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a70ce-8f87-464d-b061-6d3f19bca834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:38.400420 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9b7362c6-3884-4eb0-8ec4-a46a534bacff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:38.408096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a70ce-8f87-464d-b061-6d3f19bca834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:38.409691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a70ce-8f87-464d-b061-6d3f19bca834 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:38.413036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:38.419990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 713/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:38.421118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a70ce-8f87-464d-b061-6d3f19bca834 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:38.435119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a70ce-8f87-464d-b061-6d3f19bca834 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:38.442522 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05dee4-83ad-4c61-9ac5-e0242c3c0ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:38.443331 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05dee4-83ad-4c61-9ac5-e0242c3c0ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:38.443609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05dee4-83ad-4c61-9ac5-e0242c3c0ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:38.443881 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05dee4-83ad-4c61-9ac5-e0242c3c0ebe admin admin] PATH_INFO: `/v3/projects/dba67cb1b0b9462c90f4ca01df620284/users/bf3263d2aab24aea819859f564291fc4/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:38.495472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05dee4-83ad-4c61-9ac5-e0242c3c0ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba67cb1b0b9462c90f4ca01df620284, user_id=bf3263d2aab24aea819859f564291fc4, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:38.505487 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05dee4-83ad-4c61-9ac5-e0242c3c0ebe admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:38.584070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 722/1435] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:38.584070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f827165-d4d1-4b93-8bb4-b049b87bc298 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:38.584070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f827165-d4d1-4b93-8bb4-b049b87bc298 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:38.584070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f827165-d4d1-4b93-8bb4-b049b87bc298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:38.614143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 714/1436] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:38 2022] PUT /identity/v3/projects/dba67cb1b0b9462c90f4ca01df620284/users/bf3263d2aab24aea819859f564291fc4/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:38.614143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05dee4-83ad-4c61-9ac5-e0242c3c0ebe admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:38.617342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ab05dee4-83ad-4c61-9ac5-e0242c3c0ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:38.619641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2f827165-d4d1-4b93-8bb4-b049b87bc298 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:38.622814 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2f827165-d4d1-4b93-8bb4-b049b87bc298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:38.704021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:38.751813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05dee4-83ad-4c61-9ac5-e0242c3c0ebe admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:38.757837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 723/1437] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:38.757837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f827165-d4d1-4b93-8bb4-b049b87bc298 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:38.757837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2f827165-d4d1-4b93-8bb4-b049b87bc298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:38.765800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:38.766561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:38.766901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:38.767242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] PATH_INFO: `/v3/projects/c80f580aa50c4107b309f34df0851823` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:38.768537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80f580aa50c4107b309f34df0851823)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:38.770575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:38.846390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f827165-d4d1-4b93-8bb4-b049b87bc298 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:38.857372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa4c8-6dd6-43d9-b70d-93287107d95f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:38.857912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa4c8-6dd6-43d9-b70d-93287107d95f service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:38.858227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa4c8-6dd6-43d9-b70d-93287107d95f service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:38.858704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa4c8-6dd6-43d9-b70d-93287107d95f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:38.986275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafa4c8-6dd6-43d9-b70d-93287107d95f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:38.990657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafa4c8-6dd6-43d9-b70d-93287107d95f service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:38.994621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80f580aa50c4107b309f34df0851823', 'request_id': 'req-230f345d-3aeb-4c64-b82c-84c4c43350cf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:39.017704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 724/1438] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:39.018690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa4c8-6dd6-43d9-b70d-93287107d95f service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:39.030920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80f580aa50c4107b309f34df0851823', 'request_id': 'req-230f345d-3aeb-4c64-b82c-84c4c43350cf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:39.043449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa4c8-6dd6-43d9-b70d-93287107d95f service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:39.153429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a7ef5-a8ad-4dda-90ae-1aee242d8d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:39.154079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7ef5-a8ad-4dda-90ae-1aee242d8d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:39.154374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7ef5-a8ad-4dda-90ae-1aee242d8d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:39.154641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a7ef5-a8ad-4dda-90ae-1aee242d8d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:39.155411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a7ef5-a8ad-4dda-90ae-1aee242d8d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:39.157558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a7ef5-a8ad-4dda-90ae-1aee242d8d83 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:39.158491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80f580aa50c4107b309f34df0851823', 'request_id': 'req-230f345d-3aeb-4c64-b82c-84c4c43350cf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:39.173815 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] The token cache is being invalidate because project c80f580aa50c4107b309f34df0851823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:39.174185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:39.175222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:39.177887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 715/1439] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:38 2022] DELETE /identity/v3/projects/c80f580aa50c4107b309f34df0851823 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:39.179340 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:39.180959 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:39.196893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 725/1440] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:39.199604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a7ef5-a8ad-4dda-90ae-1aee242d8d83 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:39.205194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2c6a7ef5-a8ad-4dda-90ae-1aee242d8d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:39.222602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f345d-3aeb-4c64-b82c-84c4c43350cf admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:39.233309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b21ce-d7db-49e2-b394-e959fd9c9124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:39.234035 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b21ce-d7db-49e2-b394-e959fd9c9124 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:39.234749 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b21ce-d7db-49e2-b394-e959fd9c9124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:39.235033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b21ce-d7db-49e2-b394-e959fd9c9124 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:39.235702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b21ce-d7db-49e2-b394-e959fd9c9124 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:39.236010 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b21ce-d7db-49e2-b394-e959fd9c9124 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:39.239930 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b21ce-d7db-49e2-b394-e959fd9c9124 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:39.367556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 716/1441] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:39.369977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b21ce-d7db-49e2-b394-e959fd9c9124 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:39.380219 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ad7b21ce-d7db-49e2-b394-e959fd9c9124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:39.398975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a7ef5-a8ad-4dda-90ae-1aee242d8d83 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:39.408347 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d81b0d-ea0c-467a-bd9e-b515efddde37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:39.409076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d81b0d-ea0c-467a-bd9e-b515efddde37 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:39.409445 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d81b0d-ea0c-467a-bd9e-b515efddde37 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:39.410455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d81b0d-ea0c-467a-bd9e-b515efddde37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:39.412792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d2d81b0d-ea0c-467a-bd9e-b515efddde37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:39.533536 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b21ce-d7db-49e2-b394-e959fd9c9124 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:39.535277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d81b0d-ea0c-467a-bd9e-b515efddde37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:39.536909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d81b0d-ea0c-467a-bd9e-b515efddde37 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:39.539173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac2967-955d-416b-a8a1-19dc2d0bcfd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:39.539841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac2967-955d-416b-a8a1-19dc2d0bcfd7 service placement] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:39.540123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac2967-955d-416b-a8a1-19dc2d0bcfd7 service placement] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:39.540394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac2967-955d-416b-a8a1-19dc2d0bcfd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:39.542230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c5ac2967-955d-416b-a8a1-19dc2d0bcfd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:39.555180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 726/1442] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:39.557172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fa92e-2d08-40a4-8a32-74def5355a33 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:39.557172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fa92e-2d08-40a4-8a32-74def5355a33 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:39.557586 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fa92e-2d08-40a4-8a32-74def5355a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:39.565057 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac2967-955d-416b-a8a1-19dc2d0bcfd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:39.566977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac2967-955d-416b-a8a1-19dc2d0bcfd7 service placement] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:39.579736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 717/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:39.580567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac2967-955d-416b-a8a1-19dc2d0bcfd7 service placement] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:39.582018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c5ac2967-955d-416b-a8a1-19dc2d0bcfd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:39.596770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac2967-955d-416b-a8a1-19dc2d0bcfd7 service placement] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:39.603107 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea534f07-4771-4d84-9431-979e29825930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:39.604029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea534f07-4771-4d84-9431-979e29825930 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:39.609599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea534f07-4771-4d84-9431-979e29825930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:39.609599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea534f07-4771-4d84-9431-979e29825930 admin admin] PATH_INFO: `/v3/projects/dba67cb1b0b9462c90f4ca01df620284/users/bf3263d2aab24aea819859f564291fc4/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:39.610552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea534f07-4771-4d84-9431-979e29825930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba67cb1b0b9462c90f4ca01df620284, user_id=bf3263d2aab24aea819859f564291fc4, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:39.611939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-e58fa92e-2d08-40a4-8a32-74def5355a33 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:39.615635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea534f07-4771-4d84-9431-979e29825930 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:39.617613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e58fa92e-2d08-40a4-8a32-74def5355a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:39.650990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 727/1445] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:39.650990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fa92e-2d08-40a4-8a32-74def5355a33 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:39.653679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 718/1445] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:39 2022] PUT /identity/v3/projects/dba67cb1b0b9462c90f4ca01df620284/users/bf3263d2aab24aea819859f564291fc4/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:39.655274 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c742a-e38a-4eb8-8c07-9fa695da0099 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:39.656831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c742a-e38a-4eb8-8c07-9fa695da0099 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:39.657253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c742a-e38a-4eb8-8c07-9fa695da0099 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:39.665578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 719/1446] 10.222.0.20 () {54 vars in 961 bytes} [Wed Oct 19 02:32:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:32:39.667101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9257e9d-6717-498d-bf15-274be48401fc None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:39.670970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9257e9d-6717-498d-bf15-274be48401fc None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:39.677253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fa92e-2d08-40a4-8a32-74def5355a33 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:39.677406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9257e9d-6717-498d-bf15-274be48401fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:39.711047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e9257e9d-6717-498d-bf15-274be48401fc None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:39.723468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e9257e9d-6717-498d-bf15-274be48401fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:39.779386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:39.833046 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b04c5-f64f-466f-bcfb-9ebf96432c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:39.833856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b04c5-f64f-466f-bcfb-9ebf96432c5c admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:39.836260 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b04c5-f64f-466f-bcfb-9ebf96432c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:39.836260 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b04c5-f64f-466f-bcfb-9ebf96432c5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:39.836260 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b04c5-f64f-466f-bcfb-9ebf96432c5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:39.841568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b04c5-f64f-466f-bcfb-9ebf96432c5c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:39.865778 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 720/1447] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:39.867585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d19a7-c39f-42b6-8b9a-e8fbf9ebea89 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:39.868153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d19a7-c39f-42b6-8b9a-e8fbf9ebea89 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:39.868965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d19a7-c39f-42b6-8b9a-e8fbf9ebea89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:39.909595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-444d19a7-c39f-42b6-8b9a-e8fbf9ebea89 None None] MFA Rules not processed for user `bf3263d2aab24aea819859f564291fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:39.915742 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-444d19a7-c39f-42b6-8b9a-e8fbf9ebea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:39.943839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 728/1448] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:39 2022] POST /identity/v3/projects => generated 383 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:39.945347 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e60ce-9525-4b7c-8960-535c10663edc None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:39.945810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e60ce-9525-4b7c-8960-535c10663edc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:39.946209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e60ce-9525-4b7c-8960-535c10663edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:40.011388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-498e60ce-9525-4b7c-8960-535c10663edc None None] MFA Rules not processed for user `9243ebe24ae84157b74696c29bda3279`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:40.014555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-498e60ce-9525-4b7c-8960-535c10663edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:40.080985 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 721/1449] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:40.082092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d19a7-c39f-42b6-8b9a-e8fbf9ebea89 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:40.109197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d19a7-c39f-42b6-8b9a-e8fbf9ebea89 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:40.189374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 729/1450] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Oct 19 02:32:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:40.191240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e60ce-9525-4b7c-8960-535c10663edc None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:40.201426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd4328-7177-4738-a265-accbcfa91658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:40.201426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd4328-7177-4738-a265-accbcfa91658 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:40.201426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd4328-7177-4738-a265-accbcfa91658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:40.201851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd4328-7177-4738-a265-accbcfa91658 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:40.203967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd4328-7177-4738-a265-accbcfa91658 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:40.204946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd4328-7177-4738-a265-accbcfa91658 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:40.207882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e60ce-9525-4b7c-8960-535c10663edc None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:40.210193 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd4328-7177-4738-a265-accbcfa91658 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:40.218223 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644df9a-6433-4c6a-83d6-88918d195758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:40.218996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644df9a-6433-4c6a-83d6-88918d195758 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:40.219277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644df9a-6433-4c6a-83d6-88918d195758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:40.219713 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644df9a-6433-4c6a-83d6-88918d195758 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:40.220418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644df9a-6433-4c6a-83d6-88918d195758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:40.222807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644df9a-6433-4c6a-83d6-88918d195758 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:40.241099 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 722/1451] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:40.242688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de5514-e33d-4226-8732-5d271d190115 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:40.243519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de5514-e33d-4226-8732-5d271d190115 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:40.243828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de5514-e33d-4226-8732-5d271d190115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:40.271360 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-73de5514-e33d-4226-8732-5d271d190115 None None] MFA Rules not processed for user `d25a54e03d9848bb9747f18254dadd0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:40.271360 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-73de5514-e33d-4226-8732-5d271d190115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:40.367741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 723/1452] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:40.374366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de5514-e33d-4226-8732-5d271d190115 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:40.376848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-73de5514-e33d-4226-8732-5d271d190115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:40.394494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 730/1453] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:40 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:40.397345 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644df9a-6433-4c6a-83d6-88918d195758 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:40.456310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644df9a-6433-4c6a-83d6-88918d195758 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:40.461254 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de5514-e33d-4226-8732-5d271d190115 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:40.469078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14a048-d60a-4162-95b4-c68665ef091b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:40.470582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14a048-d60a-4162-95b4-c68665ef091b service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:40.470582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14a048-d60a-4162-95b4-c68665ef091b service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:40.470862 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14a048-d60a-4162-95b4-c68665ef091b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:40.475172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a387e6-c041-437f-b43f-5521db7ce3e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:40.476013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a387e6-c041-437f-b43f-5521db7ce3e3 service placement] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:40.476295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a387e6-c041-437f-b43f-5521db7ce3e3 service placement] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:40.476566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a387e6-c041-437f-b43f-5521db7ce3e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:40.507427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a387e6-c041-437f-b43f-5521db7ce3e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:40.509907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a387e6-c041-437f-b43f-5521db7ce3e3 service placement] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:40.552682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 731/1454] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:40.571545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a387e6-c041-437f-b43f-5521db7ce3e3 service placement] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:40.620579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a387e6-c041-437f-b43f-5521db7ce3e3 service placement] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:40.627013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691cd71-367c-41ba-aebc-da5a382d9c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:40.627912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691cd71-367c-41ba-aebc-da5a382d9c00 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:40.628818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691cd71-367c-41ba-aebc-da5a382d9c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:40.629231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691cd71-367c-41ba-aebc-da5a382d9c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:40.630004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691cd71-367c-41ba-aebc-da5a382d9c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:40.632512 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691cd71-367c-41ba-aebc-da5a382d9c00 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:40.638760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14a048-d60a-4162-95b4-c68665ef091b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:40.643464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de14a048-d60a-4162-95b4-c68665ef091b service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:40.657498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 724/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:40.658560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14a048-d60a-4162-95b4-c68665ef091b service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:40.680986 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14a048-d60a-4162-95b4-c68665ef091b service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:40.688062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4f33-1b51-4e5a-83cc-531318b3034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:40.688789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4f33-1b51-4e5a-83cc-531318b3034d admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:40.689136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4f33-1b51-4e5a-83cc-531318b3034d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:40.689526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4f33-1b51-4e5a-83cc-531318b3034d admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:40.690384 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc4f33-1b51-4e5a-83cc-531318b3034d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:40.692158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc4f33-1b51-4e5a-83cc-531318b3034d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:40.705268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 725/1456] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:40.707920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4f33-1b51-4e5a-83cc-531318b3034d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:40.729280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4f33-1b51-4e5a-83cc-531318b3034d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:40.739404 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ea3ac-8c25-417e-ab02-275ed572371b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:40.740585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ea3ac-8c25-417e-ab02-275ed572371b admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:40.740868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ea3ac-8c25-417e-ab02-275ed572371b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:40.741427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ea3ac-8c25-417e-ab02-275ed572371b admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:40.744458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ea3ac-8c25-417e-ab02-275ed572371b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:40.750577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ea3ac-8c25-417e-ab02-275ed572371b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:40.769794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 726/1457] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:32:40.779749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 732/1458] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:40 2022] POST /identity/v3/projects => generated 393 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:40.779749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691cd71-367c-41ba-aebc-da5a382d9c00 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:40.782323 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ea3ac-8c25-417e-ab02-275ed572371b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:40.786798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7f8ea3ac-8c25-417e-ab02-275ed572371b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:40.816155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ea3ac-8c25-417e-ab02-275ed572371b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:40.973159 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691cd71-367c-41ba-aebc-da5a382d9c00 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:40.977185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:40.979236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:40.979532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:40.979970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] PATH_INFO: `/v3/users/169232b5f6a84d919d0f25fcbab330aa` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:40.983629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] RBAC: Authorizing `identity:delete_user(user_id=169232b5f6a84d919d0f25fcbab330aa)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:40.991131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6422dc-ea11-4ae8-95af-353ed521e8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:40.991131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6422dc-ea11-4ae8-95af-353ed521e8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:40.991131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6422dc-ea11-4ae8-95af-353ed521e8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:40.991131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6422dc-ea11-4ae8-95af-353ed521e8a4 admin admin] PATH_INFO: `/v3/projects/ac98be80d9d74f06b1596acfbe4bfcaf/users/11acf230b01b46c995bd75de252cb5c5/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:41.005791 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:41.035229 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6422dc-ea11-4ae8-95af-353ed521e8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac98be80d9d74f06b1596acfbe4bfcaf, user_id=11acf230b01b46c995bd75de252cb5c5, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:41.049151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6422dc-ea11-4ae8-95af-353ed521e8a4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:41.166145 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 733/1459] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:40 2022] PUT /identity/v3/projects/ac98be80d9d74f06b1596acfbe4bfcaf/users/11acf230b01b46c995bd75de252cb5c5/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:41.167063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6422dc-ea11-4ae8-95af-353ed521e8a4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:41.234117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '169232b5f6a84d919d0f25fcbab330aa'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:41.234117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '169232b5f6a84d919d0f25fcbab330aa'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:41.334419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6422dc-ea11-4ae8-95af-353ed521e8a4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:41.340948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '169232b5f6a84d919d0f25fcbab330aa'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:41.341811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '169232b5f6a84d919d0f25fcbab330aa'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:41.351471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '169232b5f6a84d919d0f25fcbab330aa'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:41.363075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72618039-70bf-4bd8-94db-38815601f2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:41.366502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72618039-70bf-4bd8-94db-38815601f2fb admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:41.366793 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72618039-70bf-4bd8-94db-38815601f2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:41.368253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72618039-70bf-4bd8-94db-38815601f2fb admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:41.370632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72618039-70bf-4bd8-94db-38815601f2fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:41.381028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72618039-70bf-4bd8-94db-38815601f2fb admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:41.415794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:41.452383 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 727/1460] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:40 2022] DELETE /identity/v3/users/169232b5f6a84d919d0f25fcbab330aa => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:41.452383 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:41.452383 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:41.537757 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dd042-44fa-45a5-9079-ce036d3b694c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:41.542658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339a03f-0cf6-4a78-a6f8-855abd4ee583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:41.544329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339a03f-0cf6-4a78-a6f8-855abd4ee583 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:41.544715 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339a03f-0cf6-4a78-a6f8-855abd4ee583 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:41.545109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339a03f-0cf6-4a78-a6f8-855abd4ee583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:41.547667 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9339a03f-0cf6-4a78-a6f8-855abd4ee583 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:41.618884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339a03f-0cf6-4a78-a6f8-855abd4ee583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:41.620652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339a03f-0cf6-4a78-a6f8-855abd4ee583 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:41.637371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 728/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:41.638706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a459e79-4ec6-41dc-a67a-2083df256c65 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:41.638996 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a459e79-4ec6-41dc-a67a-2083df256c65 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:41.639600 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a459e79-4ec6-41dc-a67a-2083df256c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:41.695852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 734/1462] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:41 2022] POST /identity/v3/users => generated 297 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:41.695852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962da22-f615-4684-b683-19f3a72cf0c1 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:41.695852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962da22-f615-4684-b683-19f3a72cf0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:41.698845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962da22-f615-4684-b683-19f3a72cf0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:41.713607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-6a459e79-4ec6-41dc-a67a-2083df256c65 None None] MFA Rules not processed for user `11acf230b01b46c995bd75de252cb5c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:41.713607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6a459e79-4ec6-41dc-a67a-2083df256c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:41.743859 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-0962da22-f615-4684-b683-19f3a72cf0c1 None None] MFA Rules not processed for user `9f0fad4e9be748c786fe114895d8970c`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:41.745141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0962da22-f615-4684-b683-19f3a72cf0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:41.780719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:41.810696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 735/1463] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:41.812593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13130ec0-17e5-4e8f-b355-4cc6624f6c24 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:41.812989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13130ec0-17e5-4e8f-b355-4cc6624f6c24 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:41.813506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13130ec0-17e5-4e8f-b355-4cc6624f6c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:41.818126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 729/1464] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:41.819067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a459e79-4ec6-41dc-a67a-2083df256c65 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:41.820419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6a459e79-4ec6-41dc-a67a-2083df256c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:41.836667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-13130ec0-17e5-4e8f-b355-4cc6624f6c24 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:41.847026 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-13130ec0-17e5-4e8f-b355-4cc6624f6c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:41.919387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a459e79-4ec6-41dc-a67a-2083df256c65 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:41.925539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d461e9-06bf-4987-8677-9f12787013cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:41.926205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d461e9-06bf-4987-8677-9f12787013cb admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:41.926535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d461e9-06bf-4987-8677-9f12787013cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:41.927004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d461e9-06bf-4987-8677-9f12787013cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:41.927822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d461e9-06bf-4987-8677-9f12787013cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:41.929941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d461e9-06bf-4987-8677-9f12787013cb admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:41.932902 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 736/1465] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:41.933328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13130ec0-17e5-4e8f-b355-4cc6624f6c24 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:41.935262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-13130ec0-17e5-4e8f-b355-4cc6624f6c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:41.949276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13130ec0-17e5-4e8f-b355-4cc6624f6c24 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:41.954371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77420b-6d10-458d-b15d-20b4e1438b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:41.954992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77420b-6d10-458d-b15d-20b4e1438b64 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:41.955269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77420b-6d10-458d-b15d-20b4e1438b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:41.955540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77420b-6d10-458d-b15d-20b4e1438b64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:41.956173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e77420b-6d10-458d-b15d-20b4e1438b64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:41.958429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 730/1466] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:41.958805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e77420b-6d10-458d-b15d-20b4e1438b64 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:41.959367 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d461e9-06bf-4987-8677-9f12787013cb admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:41.961624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-53d461e9-06bf-4987-8677-9f12787013cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:42.052977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d461e9-06bf-4987-8677-9f12787013cb admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:42.071553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea240104-44f2-4c6e-9b8d-d958fc64275e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:42.072198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea240104-44f2-4c6e-9b8d-d958fc64275e service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:42.072482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea240104-44f2-4c6e-9b8d-d958fc64275e service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:42.072767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea240104-44f2-4c6e-9b8d-d958fc64275e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:42.123508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 737/1467] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:41 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:42.123508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77420b-6d10-458d-b15d-20b4e1438b64 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:42.124069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea240104-44f2-4c6e-9b8d-d958fc64275e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:42.124549 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea240104-44f2-4c6e-9b8d-d958fc64275e service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:42.161246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 731/1468] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:42.162158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea240104-44f2-4c6e-9b8d-d958fc64275e service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:42.225022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea240104-44f2-4c6e-9b8d-d958fc64275e service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:42.332410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77420b-6d10-458d-b15d-20b4e1438b64 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:42.344744 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc441e-a880-4498-8dfb-6ab24bfe36a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:42.347583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc441e-a880-4498-8dfb-6ab24bfe36a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:42.354460 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc441e-a880-4498-8dfb-6ab24bfe36a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:42.354571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc441e-a880-4498-8dfb-6ab24bfe36a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:42.355895 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc441e-a880-4498-8dfb-6ab24bfe36a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:42.357658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc441e-a880-4498-8dfb-6ab24bfe36a2 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:42.425728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 738/1469] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:42.427155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc441e-a880-4498-8dfb-6ab24bfe36a2 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:42.456382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc441e-a880-4498-8dfb-6ab24bfe36a2 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:42.472954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921799e-f867-483a-9c2f-f06903f9bc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:42.474091 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921799e-f867-483a-9c2f-f06903f9bc8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:42.474993 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921799e-f867-483a-9c2f-f06903f9bc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:42.482403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921799e-f867-483a-9c2f-f06903f9bc8b admin admin] PATH_INFO: `/v3/projects/bd2df50848dd440fb1fa9752a01dde24/users/043aaf5dfc4d4d0abc03e6566879f9f3/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:42.515196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284c121-ebd3-4b6c-84d2-74b20c47e8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:42.515995 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284c121-ebd3-4b6c-84d2-74b20c47e8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:42.516359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284c121-ebd3-4b6c-84d2-74b20c47e8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:42.516699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284c121-ebd3-4b6c-84d2-74b20c47e8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:42.517393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284c121-ebd3-4b6c-84d2-74b20c47e8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:42.524509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284c121-ebd3-4b6c-84d2-74b20c47e8a0 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:42.547217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921799e-f867-483a-9c2f-f06903f9bc8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd2df50848dd440fb1fa9752a01dde24, user_id=043aaf5dfc4d4d0abc03e6566879f9f3, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:42.550005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921799e-f867-483a-9c2f-f06903f9bc8b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:42.610803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 732/1470] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:42 2022] POST /identity/v3/users => generated 298 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:42.615081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284c121-ebd3-4b6c-84d2-74b20c47e8a0 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:42.623107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 739/1471] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:42 2022] PUT /identity/v3/projects/bd2df50848dd440fb1fa9752a01dde24/users/043aaf5dfc4d4d0abc03e6566879f9f3/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:42.635076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde2049-78dc-4b89-b85e-100685bb55e3 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:42.635606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde2049-78dc-4b89-b85e-100685bb55e3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:42.636135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde2049-78dc-4b89-b85e-100685bb55e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:42.639230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284c121-ebd3-4b6c-84d2-74b20c47e8a0 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:42.686752 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2dde2049-78dc-4b89-b85e-100685bb55e3 None None] MFA Rules not processed for user `043aaf5dfc4d4d0abc03e6566879f9f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:42.688455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2dde2049-78dc-4b89-b85e-100685bb55e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:42.771086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b63bbf-778c-4c45-b351-4855f212a309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:42.771861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b63bbf-778c-4c45-b351-4855f212a309 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:42.771993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b63bbf-778c-4c45-b351-4855f212a309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:42.772305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b63bbf-778c-4c45-b351-4855f212a309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:42.772886 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b63bbf-778c-4c45-b351-4855f212a309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:42.780138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b63bbf-778c-4c45-b351-4855f212a309 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:42.784500 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 740/1472] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:42.790484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde2049-78dc-4b89-b85e-100685bb55e3 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:42.809691 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde2049-78dc-4b89-b85e-100685bb55e3 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:42.821200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 733/1473] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:32:42.826142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b63bbf-778c-4c45-b351-4855f212a309 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:42.837574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0919ea2-a2ae-4496-8061-2dabb81511de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:42.838282 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0919ea2-a2ae-4496-8061-2dabb81511de admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:42.838511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0919ea2-a2ae-4496-8061-2dabb81511de admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:42.839157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0919ea2-a2ae-4496-8061-2dabb81511de admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:42.839828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0919ea2-a2ae-4496-8061-2dabb81511de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:42.849758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0919ea2-a2ae-4496-8061-2dabb81511de admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:42.849972 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b63bbf-778c-4c45-b351-4855f212a309 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:42.866124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8a901-d200-4eea-9b12-b38c0d8ef600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:42.866818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8a901-d200-4eea-9b12-b38c0d8ef600 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:42.867095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8a901-d200-4eea-9b12-b38c0d8ef600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:42.867372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d8a901-d200-4eea-9b12-b38c0d8ef600 admin admin] PATH_INFO: `/v3/projects/49e64ba84d66400ea671b835030c7abf/users/4076ab33f5b64b9b80c34e5406c9fb91/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:42.891558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8a901-d200-4eea-9b12-b38c0d8ef600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e64ba84d66400ea671b835030c7abf, user_id=4076ab33f5b64b9b80c34e5406c9fb91, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:42.894342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d8a901-d200-4eea-9b12-b38c0d8ef600 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:43.008074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 734/1474] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:42 2022] PUT /identity/v3/projects/49e64ba84d66400ea671b835030c7abf/users/4076ab33f5b64b9b80c34e5406c9fb91/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:43.009485 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 741/1475] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:42 2022] POST /identity/v3/projects => generated 391 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:43.013707 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0919ea2-a2ae-4496-8061-2dabb81511de admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:43.013852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8a901-d200-4eea-9b12-b38c0d8ef600 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:43.158285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d8a901-d200-4eea-9b12-b38c0d8ef600 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:43.165490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bd840-61b4-4cd4-8cc4-704a6f4ac18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:43.166447 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd840-61b4-4cd4-8cc4-704a6f4ac18f admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:43.166896 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd840-61b4-4cd4-8cc4-704a6f4ac18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:43.167386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78bd840-61b4-4cd4-8cc4-704a6f4ac18f admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:43.168310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78bd840-61b4-4cd4-8cc4-704a6f4ac18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:43.170873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78bd840-61b4-4cd4-8cc4-704a6f4ac18f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:43.172384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0919ea2-a2ae-4496-8061-2dabb81511de admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:43.189027 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856b15b-1fba-47dd-93cb-9730e0984a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:43.189027 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856b15b-1fba-47dd-93cb-9730e0984a63 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:43.189027 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856b15b-1fba-47dd-93cb-9730e0984a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:43.189027 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856b15b-1fba-47dd-93cb-9730e0984a63 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:43.189432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856b15b-1fba-47dd-93cb-9730e0984a63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:43.192016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856b15b-1fba-47dd-93cb-9730e0984a63 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:43.207967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 735/1476] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:43.208965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bd840-61b4-4cd4-8cc4-704a6f4ac18f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:43.212693 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e78bd840-61b4-4cd4-8cc4-704a6f4ac18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:43.232001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78bd840-61b4-4cd4-8cc4-704a6f4ac18f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:43.247469 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:43.248183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:43.248467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:43.248739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] PATH_INFO: `/v3/projects/0a4ca42d6f1e4e4e92b778ae0a907d5f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:43.251113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a4ca42d6f1e4e4e92b778ae0a907d5f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:43.264983 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:43.325804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 742/1477] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:43 2022] POST /identity/v3/users => generated 302 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:43.329490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856b15b-1fba-47dd-93cb-9730e0984a63 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:43.351255 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856b15b-1fba-47dd-93cb-9730e0984a63 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:43.362296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a7263-07a0-4e27-af8e-7c4fbf27c7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:43.364033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a7263-07a0-4e27-af8e-7c4fbf27c7fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:43.365845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a7263-07a0-4e27-af8e-7c4fbf27c7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:43.368960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a7263-07a0-4e27-af8e-7c4fbf27c7fa admin admin] PATH_INFO: `/v3/projects/49e64ba84d66400ea671b835030c7abf/users/4076ab33f5b64b9b80c34e5406c9fb91/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:43.371375 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a7263-07a0-4e27-af8e-7c4fbf27c7fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e64ba84d66400ea671b835030c7abf, user_id=4076ab33f5b64b9b80c34e5406c9fb91, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:43.376960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a7263-07a0-4e27-af8e-7c4fbf27c7fa admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:43.407051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 743/1478] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:43 2022] PUT /identity/v3/projects/49e64ba84d66400ea671b835030c7abf/users/4076ab33f5b64b9b80c34e5406c9fb91/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:43.408078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a7263-07a0-4e27-af8e-7c4fbf27c7fa admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:43.441359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a4ca42d6f1e4e4e92b778ae0a907d5f', 'request_id': 'req-efcddbec-c949-4d70-ba03-468fc8ff1e23'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:43.491377 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a4ca42d6f1e4e4e92b778ae0a907d5f', 'request_id': 'req-efcddbec-c949-4d70-ba03-468fc8ff1e23'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:43.527414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a7263-07a0-4e27-af8e-7c4fbf27c7fa admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:43.580529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a4ca42d6f1e4e4e92b778ae0a907d5f', 'request_id': 'req-efcddbec-c949-4d70-ba03-468fc8ff1e23'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:43.597859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] The token cache is being invalidate because project 0a4ca42d6f1e4e4e92b778ae0a907d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:43.598599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:43.603041 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-efcddbec-c949-4d70-ba03-468fc8ff1e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:43.607126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 736/1479] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:43 2022] DELETE /identity/v3/projects/0a4ca42d6f1e4e4e92b778ae0a907d5f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:43.609506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc76a-f34c-480e-b7ec-4a7f25b36625 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:43.610178 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc76a-f34c-480e-b7ec-4a7f25b36625 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:43.610559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdc76a-f34c-480e-b7ec-4a7f25b36625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:43.670105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-39fdc76a-f34c-480e-b7ec-4a7f25b36625 None None] MFA Rules not processed for user `4076ab33f5b64b9b80c34e5406c9fb91`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:43.675120 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-39fdc76a-f34c-480e-b7ec-4a7f25b36625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:43.711848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33821d61-2aba-43ee-92fd-287eecb2a07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:43.712773 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821d61-2aba-43ee-92fd-287eecb2a07f admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:43.713247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821d61-2aba-43ee-92fd-287eecb2a07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:43.713696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821d61-2aba-43ee-92fd-287eecb2a07f admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:43.714552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33821d61-2aba-43ee-92fd-287eecb2a07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:43.716382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33821d61-2aba-43ee-92fd-287eecb2a07f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:43.786504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 744/1480] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:43.786504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33821d61-2aba-43ee-92fd-287eecb2a07f admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:43.790222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-33821d61-2aba-43ee-92fd-287eecb2a07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:43.822356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33821d61-2aba-43ee-92fd-287eecb2a07f admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:43.827930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:43.828818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:43.829090 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:43.829533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] PATH_INFO: `/v3/users/922fec5eef034f9f93569d4c59efac60` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:43.831367 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=922fec5eef034f9f93569d4c59efac60)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:43.834635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:43.861221 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 737/1481] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:43.861812 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc76a-f34c-480e-b7ec-4a7f25b36625 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:43.867244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-39fdc76a-f34c-480e-b7ec-4a7f25b36625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:43.901133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdc76a-f34c-480e-b7ec-4a7f25b36625 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:43.908723 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:43.932781 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641a86-66a1-4c36-8192-d1ed06e087b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:43.933470 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641a86-66a1-4c36-8192-d1ed06e087b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:43.934019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641a86-66a1-4c36-8192-d1ed06e087b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:43.934019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5641a86-66a1-4c36-8192-d1ed06e087b2 admin admin] PATH_INFO: `/v3/projects/63c0a11ce0544fc481b92e2462f833f8/users/8a3650c2cae941428bb00d916f1c9011/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:43.961150 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5641a86-66a1-4c36-8192-d1ed06e087b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c0a11ce0544fc481b92e2462f833f8, user_id=8a3650c2cae941428bb00d916f1c9011, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:43.963882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5641a86-66a1-4c36-8192-d1ed06e087b2 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:44.060822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 738/1482] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:43 2022] PUT /identity/v3/projects/63c0a11ce0544fc481b92e2462f833f8/users/8a3650c2cae941428bb00d916f1c9011/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:44.061751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641a86-66a1-4c36-8192-d1ed06e087b2 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:44.064148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f5641a86-66a1-4c36-8192-d1ed06e087b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:44.094474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:44.126321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922fec5eef034f9f93569d4c59efac60'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:44.126321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922fec5eef034f9f93569d4c59efac60'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:44.132628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922fec5eef034f9f93569d4c59efac60'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:44.162164 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5641a86-66a1-4c36-8192-d1ed06e087b2 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:44.167031 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a874d-d84c-46a2-9b26-f7ec70a5412b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:44.167640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a874d-d84c-46a2-9b26-f7ec70a5412b service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:44.167913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a874d-d84c-46a2-9b26-f7ec70a5412b service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:44.168582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a874d-d84c-46a2-9b26-f7ec70a5412b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:44.173028 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d12a874d-d84c-46a2-9b26-f7ec70a5412b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:44.248001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a874d-d84c-46a2-9b26-f7ec70a5412b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:44.249569 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a874d-d84c-46a2-9b26-f7ec70a5412b service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:44.264329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 739/1483] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:44.264329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a874d-d84c-46a2-9b26-f7ec70a5412b service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:44.265467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d12a874d-d84c-46a2-9b26-f7ec70a5412b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:44.322122 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922fec5eef034f9f93569d4c59efac60'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:44.322763 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922fec5eef034f9f93569d4c59efac60'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:44.368937 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 745/1484] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:43 2022] DELETE /identity/v3/users/922fec5eef034f9f93569d4c59efac60 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:44.415554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a874d-d84c-46a2-9b26-f7ec70a5412b service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:44.458494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:44.460208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:44.551646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa97132-74d3-45ef-bbd3-1ebcffb036c6 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:44.559860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c447e1-7155-4107-aeea-c4ad8892a025 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:44.560719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c447e1-7155-4107-aeea-c4ad8892a025 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:44.561306 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c447e1-7155-4107-aeea-c4ad8892a025 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:44.562675 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c447e1-7155-4107-aeea-c4ad8892a025 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:44.565262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-58c447e1-7155-4107-aeea-c4ad8892a025 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:44.589771 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535fff7-42fc-49f3-9810-555545b94fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:44.590668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535fff7-42fc-49f3-9810-555545b94fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:44.591270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535fff7-42fc-49f3-9810-555545b94fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:44.591622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535fff7-42fc-49f3-9810-555545b94fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:44.592735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535fff7-42fc-49f3-9810-555545b94fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:44.594771 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535fff7-42fc-49f3-9810-555545b94fe2 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:44.607156 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 740/1485] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:44.611121 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535fff7-42fc-49f3-9810-555545b94fe2 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:44.625048 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535fff7-42fc-49f3-9810-555545b94fe2 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:44.630353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669648f9-076b-4cae-a045-1ae4bad6f310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:44.631364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669648f9-076b-4cae-a045-1ae4bad6f310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:44.631364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669648f9-076b-4cae-a045-1ae4bad6f310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:44.631639 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669648f9-076b-4cae-a045-1ae4bad6f310 admin admin] PATH_INFO: `/v3/projects/63c0a11ce0544fc481b92e2462f833f8/users/8a3650c2cae941428bb00d916f1c9011/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:44.633786 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669648f9-076b-4cae-a045-1ae4bad6f310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63c0a11ce0544fc481b92e2462f833f8, user_id=8a3650c2cae941428bb00d916f1c9011, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:44.636797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669648f9-076b-4cae-a045-1ae4bad6f310 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:44.649651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 741/1486] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:44 2022] PUT /identity/v3/projects/63c0a11ce0544fc481b92e2462f833f8/users/8a3650c2cae941428bb00d916f1c9011/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:44.652910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c447e1-7155-4107-aeea-c4ad8892a025 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:44.655219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c447e1-7155-4107-aeea-c4ad8892a025 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:44.661616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bcc2f-0899-4ae5-9441-c7995a9032b7 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:44.661899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bcc2f-0899-4ae5-9441-c7995a9032b7 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:44.662215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bcc2f-0899-4ae5-9441-c7995a9032b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:44.665869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 746/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:32:44.694983 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-2b4bcc2f-0899-4ae5-9441-c7995a9032b7 None None] MFA Rules not processed for user `8a3650c2cae941428bb00d916f1c9011`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:44.696459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2b4bcc2f-0899-4ae5-9441-c7995a9032b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:44.877324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 742/1488] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:44 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:44.955746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c447e1-7155-4107-aeea-c4ad8892a025 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:44.957934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-58c447e1-7155-4107-aeea-c4ad8892a025 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:44.980971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c447e1-7155-4107-aeea-c4ad8892a025 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:45.051217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b2285-447c-4336-821c-43bbdabe8615 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:45.052072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b2285-447c-4336-821c-43bbdabe8615 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:45.052428 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b2285-447c-4336-821c-43bbdabe8615 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:45.052779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b2285-447c-4336-821c-43bbdabe8615 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:45.083643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b2285-447c-4336-821c-43bbdabe8615 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:45.086952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b2285-447c-4336-821c-43bbdabe8615 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:45.109372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 747/1489] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:44 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:45.165740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bcc2f-0899-4ae5-9441-c7995a9032b7 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:45.176533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bcc2f-0899-4ae5-9441-c7995a9032b7 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:45.181027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ab909-7e86-41ac-babc-27ec7e0148f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:45.181647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ab909-7e86-41ac-babc-27ec7e0148f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:45.181919 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ab909-7e86-41ac-babc-27ec7e0148f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:45.182217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ab909-7e86-41ac-babc-27ec7e0148f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:45.184007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-772ab909-7e86-41ac-babc-27ec7e0148f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:45.216791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b2285-447c-4336-821c-43bbdabe8615 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:45.218463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-994b2285-447c-4336-821c-43bbdabe8615 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:45.285303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ab909-7e86-41ac-babc-27ec7e0148f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:45.286833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ab909-7e86-41ac-babc-27ec7e0148f0 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:45.316980 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 743/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:45.324181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b2285-447c-4336-821c-43bbdabe8615 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:45.329425 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:45.330156 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:45.330475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:45.330823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] PATH_INFO: `/v3/projects/d7ef6b1951f64104acf8414fc1f31cc5` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:45.332065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ef6b1951f64104acf8414fc1f31cc5)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:45.347545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:45.351956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:45.535806 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486d794-c3ad-4533-8f5e-2620cb66a3d8 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:45.536012 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486d794-c3ad-4533-8f5e-2620cb66a3d8 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:45.536284 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486d794-c3ad-4533-8f5e-2620cb66a3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:45.562197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ef6b1951f64104acf8414fc1f31cc5', 'request_id': 'req-f14bf32f-7572-4e09-9518-9f9fc1adfb48'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:45.572733 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ef6b1951f64104acf8414fc1f31cc5', 'request_id': 'req-f14bf32f-7572-4e09-9518-9f9fc1adfb48'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:45.608895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-5486d794-c3ad-4533-8f5e-2620cb66a3d8 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:45.610159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5486d794-c3ad-4533-8f5e-2620cb66a3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:45.724146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ef6b1951f64104acf8414fc1f31cc5', 'request_id': 'req-f14bf32f-7572-4e09-9518-9f9fc1adfb48'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:45.742394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] The token cache is being invalidate because project d7ef6b1951f64104acf8414fc1f31cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:45.742690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:45.743525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f14bf32f-7572-4e09-9518-9f9fc1adfb48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:45.745641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 748/1491] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:45 2022] DELETE /identity/v3/projects/d7ef6b1951f64104acf8414fc1f31cc5 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:45.751149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcde95-7906-45f1-8d23-96fe906da705 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:45.751437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcde95-7906-45f1-8d23-96fe906da705 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:45.751909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcde95-7906-45f1-8d23-96fe906da705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:45.759355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 744/1492] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:32:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:45.802799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-86bcde95-7906-45f1-8d23-96fe906da705 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:45.802799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-86bcde95-7906-45f1-8d23-96fe906da705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:45.895332 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 749/1493] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:45.896890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eeb8c-792e-4d8f-89d1-f72f1b74e078 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:45.897179 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eeb8c-792e-4d8f-89d1-f72f1b74e078 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:45.897473 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9eeb8c-792e-4d8f-89d1-f72f1b74e078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:45.917176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-7c9eeb8c-792e-4d8f-89d1-f72f1b74e078 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:45.918403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7c9eeb8c-792e-4d8f-89d1-f72f1b74e078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:45.927159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 745/1494] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:45.931431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcde95-7906-45f1-8d23-96fe906da705 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:45.942187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bcde95-7906-45f1-8d23-96fe906da705 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:45.946985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da03487-c648-44b9-824b-8e89083f2a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:45.947598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da03487-c648-44b9-824b-8e89083f2a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:45.947879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da03487-c648-44b9-824b-8e89083f2a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:45.948186 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da03487-c648-44b9-824b-8e89083f2a13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:45.948843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da03487-c648-44b9-824b-8e89083f2a13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:45.949562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da03487-c648-44b9-824b-8e89083f2a13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:45.950821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da03487-c648-44b9-824b-8e89083f2a13 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:45.964357 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 750/1495] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:32:45.976004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9eeb8c-792e-4d8f-89d1-f72f1b74e078 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:46.011753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9eeb8c-792e-4d8f-89d1-f72f1b74e078 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:46.038840 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4b748-3e94-4ce2-b018-fdfc0f1dab47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:46.039495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b748-3e94-4ce2-b018-fdfc0f1dab47 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:46.039776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b748-3e94-4ce2-b018-fdfc0f1dab47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:46.040046 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b748-3e94-4ce2-b018-fdfc0f1dab47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:46.040697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4b748-3e94-4ce2-b018-fdfc0f1dab47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:46.042847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4b748-3e94-4ce2-b018-fdfc0f1dab47 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:46.198454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 746/1496] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:45 2022] POST /identity/v3/projects => generated 365 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:46.203405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da03487-c648-44b9-824b-8e89083f2a13 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:46.217269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da03487-c648-44b9-824b-8e89083f2a13 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:46.249889 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4b748-3e94-4ce2-b018-fdfc0f1dab47 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:46.257769 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-49d4b748-3e94-4ce2-b018-fdfc0f1dab47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:46.321654 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:46.349234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e76ea1-ffdc-4dfa-a740-da7890b9b490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:46.349963 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e76ea1-ffdc-4dfa-a740-da7890b9b490 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:46.350330 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e76ea1-ffdc-4dfa-a740-da7890b9b490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:46.350757 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e76ea1-ffdc-4dfa-a740-da7890b9b490 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:46.351517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e76ea1-ffdc-4dfa-a740-da7890b9b490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:46.353582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e76ea1-ffdc-4dfa-a740-da7890b9b490 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:46.393243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:46.517690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 751/1497] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:46 2022] POST /identity/v3/users => generated 283 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:32:46.523219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e76ea1-ffdc-4dfa-a740-da7890b9b490 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:46.549849 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e76ea1-ffdc-4dfa-a740-da7890b9b490 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:46.563610 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4b748-3e94-4ce2-b018-fdfc0f1dab47 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:46.570031 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef291de-e12c-400b-83cf-555a924ece60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:46.570812 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef291de-e12c-400b-83cf-555a924ece60 service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:46.571258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef291de-e12c-400b-83cf-555a924ece60 service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:46.571540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef291de-e12c-400b-83cf-555a924ece60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:46.579913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8ef291de-e12c-400b-83cf-555a924ece60 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:46.585179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3739fc7-78aa-4e1a-8082-7b3422e8a1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:46.588539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739fc7-78aa-4e1a-8082-7b3422e8a1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:46.588539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739fc7-78aa-4e1a-8082-7b3422e8a1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:46.588539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3739fc7-78aa-4e1a-8082-7b3422e8a1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:46.588539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3739fc7-78aa-4e1a-8082-7b3422e8a1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:46.597724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3739fc7-78aa-4e1a-8082-7b3422e8a1d6 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:46.645910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 752/1498] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:46.653778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3739fc7-78aa-4e1a-8082-7b3422e8a1d6 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:46.707076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3739fc7-78aa-4e1a-8082-7b3422e8a1d6 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:46.722242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a19073-9f4e-4400-a90d-13d47cf231b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:46.726625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a19073-9f4e-4400-a90d-13d47cf231b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:46.726625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a19073-9f4e-4400-a90d-13d47cf231b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:46.726625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a19073-9f4e-4400-a90d-13d47cf231b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:46.729434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef291de-e12c-400b-83cf-555a924ece60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:46.732249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a19073-9f4e-4400-a90d-13d47cf231b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:46.733379 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef291de-e12c-400b-83cf-555a924ece60 service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:46.734199 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a19073-9f4e-4400-a90d-13d47cf231b9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:46.750320 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 747/1499] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:32:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:32:46.781834 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 753/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:46.781937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef291de-e12c-400b-83cf-555a924ece60 service swift] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:46.813167 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef291de-e12c-400b-83cf-555a924ece60 service swift] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:46.830507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a79879-3799-4ca0-bd5d-923fe09bf10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:46.832616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a79879-3799-4ca0-bd5d-923fe09bf10a admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:46.833148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a79879-3799-4ca0-bd5d-923fe09bf10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:46.833629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a79879-3799-4ca0-bd5d-923fe09bf10a admin admin] PATH_INFO: `/v3/projects/3b20a380c6e6487e805f4d011a00a423/users/988fa9f598bd4836a7ff88cee07c717f/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:46.874440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a79879-3799-4ca0-bd5d-923fe09bf10a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b20a380c6e6487e805f4d011a00a423, user_id=988fa9f598bd4836a7ff88cee07c717f, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:46.913681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a79879-3799-4ca0-bd5d-923fe09bf10a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:46.946481 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f17b1f-feb8-4863-b94c-4a92dab51f30 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:46.947010 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f17b1f-feb8-4863-b94c-4a92dab51f30 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:46.947488 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f17b1f-feb8-4863-b94c-4a92dab51f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:46.972117 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-52f17b1f-feb8-4863-b94c-4a92dab51f30 None None] MFA Rules not processed for user `b5ff8583997843258e315413abac172f`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:46.972372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-52f17b1f-feb8-4863-b94c-4a92dab51f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:47.051385 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 754/1501] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Oct 19 02:32:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:47.071289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 748/1502] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:46 2022] PUT /identity/v3/projects/3b20a380c6e6487e805f4d011a00a423/users/988fa9f598bd4836a7ff88cee07c717f/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:47.081987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933aea8-9401-4e13-883d-896355e75360 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:47.098999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933aea8-9401-4e13-883d-896355e75360 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:47.099431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933aea8-9401-4e13-883d-896355e75360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:47.196360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-b933aea8-9401-4e13-883d-896355e75360 None None] MFA Rules not processed for user `988fa9f598bd4836a7ff88cee07c717f`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:47.200966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b933aea8-9401-4e13-883d-896355e75360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:47.212241 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a79879-3799-4ca0-bd5d-923fe09bf10a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:47.215456 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-47a79879-3799-4ca0-bd5d-923fe09bf10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:47.439830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a79879-3799-4ca0-bd5d-923fe09bf10a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:47.448118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05aaa2a-2054-4db0-aed6-ce15e7db1b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:47.449215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05aaa2a-2054-4db0-aed6-ce15e7db1b7f service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:47.449507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05aaa2a-2054-4db0-aed6-ce15e7db1b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:47.449782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05aaa2a-2054-4db0-aed6-ce15e7db1b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:47.452997 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 755/1503] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:47.454123 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933aea8-9401-4e13-883d-896355e75360 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:47.463653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05aaa2a-2054-4db0-aed6-ce15e7db1b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:47.465128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05aaa2a-2054-4db0-aed6-ce15e7db1b7f service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:47.476381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 749/1504] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:47.571871 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933aea8-9401-4e13-883d-896355e75360 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:47.576870 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad5edb-8934-4738-b290-32abd2fc4f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:47.579247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5edb-8934-4738-b290-32abd2fc4f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:47.579247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5edb-8934-4738-b290-32abd2fc4f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:47.579247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5edb-8934-4738-b290-32abd2fc4f39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:47.579247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad5edb-8934-4738-b290-32abd2fc4f39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:47.580759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad5edb-8934-4738-b290-32abd2fc4f39 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:47.739346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 756/1505] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:47 2022] POST /identity/v3/projects => generated 365 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:47.739648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05aaa2a-2054-4db0-aed6-ce15e7db1b7f service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:47.764274 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05aaa2a-2054-4db0-aed6-ce15e7db1b7f service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:47.851495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965f96c-7016-4afb-983e-d7c45d0293e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:47.853188 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965f96c-7016-4afb-983e-d7c45d0293e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:47.853634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965f96c-7016-4afb-983e-d7c45d0293e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:47.853912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965f96c-7016-4afb-983e-d7c45d0293e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:47.854694 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965f96c-7016-4afb-983e-d7c45d0293e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:47.857533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965f96c-7016-4afb-983e-d7c45d0293e2 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:47.993386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 750/1506] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:47 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:47.997940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad5edb-8934-4738-b290-32abd2fc4f39 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:48.009345 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad5edb-8934-4738-b290-32abd2fc4f39 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:48.015319 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e27db-7c25-4ee1-a16c-e5a60b638e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:48.016072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e27db-7c25-4ee1-a16c-e5a60b638e06 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:48.016547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e27db-7c25-4ee1-a16c-e5a60b638e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:48.016913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e27db-7c25-4ee1-a16c-e5a60b638e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:48.017681 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e27db-7c25-4ee1-a16c-e5a60b638e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:48.019451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e27db-7c25-4ee1-a16c-e5a60b638e06 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:48.030055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 757/1507] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:48.034379 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965f96c-7016-4afb-983e-d7c45d0293e2 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:48.046858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965f96c-7016-4afb-983e-d7c45d0293e2 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:48.051933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988780f-16f6-4691-a453-dccb0f3036a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:48.052610 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988780f-16f6-4691-a453-dccb0f3036a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:48.052889 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988780f-16f6-4691-a453-dccb0f3036a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:48.053166 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988780f-16f6-4691-a453-dccb0f3036a5 admin admin] PATH_INFO: `/v3/projects/82753ee6eec04972a1888eecf140eb24/users/35689fe0ec754caf8be9a351f110cc3f/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:48.079053 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988780f-16f6-4691-a453-dccb0f3036a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82753ee6eec04972a1888eecf140eb24, user_id=35689fe0ec754caf8be9a351f110cc3f, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:48.083686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988780f-16f6-4691-a453-dccb0f3036a5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:48.147587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:48.223845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 751/1508] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:48 2022] PUT /identity/v3/projects/82753ee6eec04972a1888eecf140eb24/users/35689fe0ec754caf8be9a351f110cc3f/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:48.230337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e27db-7c25-4ee1-a16c-e5a60b638e06 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:48.244464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e27db-7c25-4ee1-a16c-e5a60b638e06 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:48.329008 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b613d71-f0d5-4b43-8388-b1d4ec05aea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:48.330003 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613d71-f0d5-4b43-8388-b1d4ec05aea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:48.330451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613d71-f0d5-4b43-8388-b1d4ec05aea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:48.331271 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b613d71-f0d5-4b43-8388-b1d4ec05aea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:48.332065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b613d71-f0d5-4b43-8388-b1d4ec05aea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:48.333824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b613d71-f0d5-4b43-8388-b1d4ec05aea2 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:48.347244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 758/1509] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:48.351873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988780f-16f6-4691-a453-dccb0f3036a5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:48.364683 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988780f-16f6-4691-a453-dccb0f3036a5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:48.369911 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc39980-f03d-40bd-b697-6cb0edfe450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:48.376027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc39980-f03d-40bd-b697-6cb0edfe450e admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:48.376027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc39980-f03d-40bd-b697-6cb0edfe450e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:48.376027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc39980-f03d-40bd-b697-6cb0edfe450e admin admin] PATH_INFO: `/v3/projects/82753ee6eec04972a1888eecf140eb24/users/35689fe0ec754caf8be9a351f110cc3f/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:48.376027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc39980-f03d-40bd-b697-6cb0edfe450e admin admin] RBAC: Authorizing `identity:create_grant(project_id=82753ee6eec04972a1888eecf140eb24, user_id=35689fe0ec754caf8be9a351f110cc3f, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:48.377462 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc39980-f03d-40bd-b697-6cb0edfe450e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:48.389761 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 752/1510] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:48 2022] PUT /identity/v3/projects/82753ee6eec04972a1888eecf140eb24/users/35689fe0ec754caf8be9a351f110cc3f/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:48.400424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8da03-87d6-4fc6-abbb-0b512aee40a9 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:48.400704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8da03-87d6-4fc6-abbb-0b512aee40a9 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:48.400973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8da03-87d6-4fc6-abbb-0b512aee40a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:48.430569 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-d3a8da03-87d6-4fc6-abbb-0b512aee40a9 None None] MFA Rules not processed for user `35689fe0ec754caf8be9a351f110cc3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:48.431845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d3a8da03-87d6-4fc6-abbb-0b512aee40a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:48.516420 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 759/1511] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:48.530145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc39980-f03d-40bd-b697-6cb0edfe450e admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:48.534085 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2dc39980-f03d-40bd-b697-6cb0edfe450e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:48.611883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc39980-f03d-40bd-b697-6cb0edfe450e admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:48.615937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e3c9-2d95-4147-8fcd-375fd29e3a65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:48.616483 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e3c9-2d95-4147-8fcd-375fd29e3a65 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:48.616759 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e3c9-2d95-4147-8fcd-375fd29e3a65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:48.617024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e3c9-2d95-4147-8fcd-375fd29e3a65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:48.627208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0e3c9-2d95-4147-8fcd-375fd29e3a65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:48.628646 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0e3c9-2d95-4147-8fcd-375fd29e3a65 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:48.638435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 753/1512] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:48.911278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8da03-87d6-4fc6-abbb-0b512aee40a9 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:48.921180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8da03-87d6-4fc6-abbb-0b512aee40a9 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:48.924757 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95626667-2d11-48db-bda9-6132be10ef58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:48.925360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95626667-2d11-48db-bda9-6132be10ef58 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:48.925638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95626667-2d11-48db-bda9-6132be10ef58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:48.925989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95626667-2d11-48db-bda9-6132be10ef58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:48.928109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4543e-9cfb-4e63-9262-9108276c6a66 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:48.929077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4543e-9cfb-4e63-9262-9108276c6a66 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:48.929579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4543e-9cfb-4e63-9262-9108276c6a66 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:48.931670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 754/1513] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:32:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:32:48.939256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d2653-69a9-4119-9367-b97e8aa9eb55 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:48.939979 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d2653-69a9-4119-9367-b97e8aa9eb55 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:48.940483 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d2653-69a9-4119-9367-b97e8aa9eb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:48.968152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-a43d2653-69a9-4119-9367-b97e8aa9eb55 None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:48.969430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a43d2653-69a9-4119-9367-b97e8aa9eb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:48.986611 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95626667-2d11-48db-bda9-6132be10ef58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:48.988419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95626667-2d11-48db-bda9-6132be10ef58 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:48.999084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 760/1514] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:32:49.045578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 755/1515] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:32:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:49.056843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:49.058200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95626667-2d11-48db-bda9-6132be10ef58 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:49.060138 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-95626667-2d11-48db-bda9-6132be10ef58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:49.076516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95626667-2d11-48db-bda9-6132be10ef58 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:49.085798 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6395606-c1b9-4381-a7b2-8fe232ab0936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:49.086459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6395606-c1b9-4381-a7b2-8fe232ab0936 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:49.086740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6395606-c1b9-4381-a7b2-8fe232ab0936 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:49.087013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6395606-c1b9-4381-a7b2-8fe232ab0936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:49.096796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6395606-c1b9-4381-a7b2-8fe232ab0936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:49.099178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6395606-c1b9-4381-a7b2-8fe232ab0936 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:49.113322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 761/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:49.747841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:50.696156 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:50.837793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d2653-69a9-4119-9367-b97e8aa9eb55 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:50.839750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a43d2653-69a9-4119-9367-b97e8aa9eb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:50.867415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:50.939114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d2653-69a9-4119-9367-b97e8aa9eb55 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:50.943879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:50.944847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:50.945286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:50.945712 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] PATH_INFO: `/v3/users/11acf230b01b46c995bd75de252cb5c5` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:50.947059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] RBAC: Authorizing `identity:delete_user(user_id=11acf230b01b46c995bd75de252cb5c5)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:50.949205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:51.382621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11acf230b01b46c995bd75de252cb5c5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:51.382621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11acf230b01b46c995bd75de252cb5c5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:51.484303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11acf230b01b46c995bd75de252cb5c5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:51.485003 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11acf230b01b46c995bd75de252cb5c5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:51.509436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11acf230b01b46c995bd75de252cb5c5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:51.534883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 756/1517] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:50 2022] DELETE /identity/v3/users/11acf230b01b46c995bd75de252cb5c5 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:51.536473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73740c39-1d71-45fd-99cd-39500b591eef None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:51.536938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73740c39-1d71-45fd-99cd-39500b591eef None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:51.537363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73740c39-1d71-45fd-99cd-39500b591eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:51.568387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-73740c39-1d71-45fd-99cd-39500b591eef None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:51.569909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-73740c39-1d71-45fd-99cd-39500b591eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:51.743079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:51.744595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:51.746726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 762/1518] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:51.845090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3428cb-e76a-416e-9d03-3412c1c187be admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:51.848665 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa8eca-af0c-427f-886a-d5758e75bf5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:51.849292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa8eca-af0c-427f-886a-d5758e75bf5f service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:51.849599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa8eca-af0c-427f-886a-d5758e75bf5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:51.850026 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa8eca-af0c-427f-886a-d5758e75bf5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:51.889384 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa8eca-af0c-427f-886a-d5758e75bf5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:51.891046 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa8eca-af0c-427f-886a-d5758e75bf5f service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:51.909173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 757/1519] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:32:52.137109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:52.215748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:52.443324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73740c39-1d71-45fd-99cd-39500b591eef None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:52.444688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-73740c39-1d71-45fd-99cd-39500b591eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:52.457888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73740c39-1d71-45fd-99cd-39500b591eef None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:52.462711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c22c6-ffad-48bb-9be3-aefd6985e29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:52.463272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c22c6-ffad-48bb-9be3-aefd6985e29d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:52.463580 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c22c6-ffad-48bb-9be3-aefd6985e29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:52.463850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c22c6-ffad-48bb-9be3-aefd6985e29d admin admin] PATH_INFO: `/v3/users/043aaf5dfc4d4d0abc03e6566879f9f3` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:52.465044 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c22c6-ffad-48bb-9be3-aefd6985e29d admin admin] RBAC: Authorizing `identity:delete_user(user_id=043aaf5dfc4d4d0abc03e6566879f9f3)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:52.467125 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c22c6-ffad-48bb-9be3-aefd6985e29d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:52.546780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa8eca-af0c-427f-886a-d5758e75bf5f service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:52.550270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-65aa8eca-af0c-427f-886a-d5758e75bf5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:52.571253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa8eca-af0c-427f-886a-d5758e75bf5f service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:52.577830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:52.586853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:52.587330 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:52.587611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] PATH_INFO: `/v3/projects/ac98be80d9d74f06b1596acfbe4bfcaf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:52.589363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac98be80d9d74f06b1596acfbe4bfcaf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:52.593030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:52.692257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a39c22c6-ffad-48bb-9be3-aefd6985e29d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '043aaf5dfc4d4d0abc03e6566879f9f3'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:52.694274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a39c22c6-ffad-48bb-9be3-aefd6985e29d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '043aaf5dfc4d4d0abc03e6566879f9f3'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:52.703699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:52.706446 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a39c22c6-ffad-48bb-9be3-aefd6985e29d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '043aaf5dfc4d4d0abc03e6566879f9f3'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:52.738818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac98be80d9d74f06b1596acfbe4bfcaf', 'request_id': 'req-8a9b6f99-b342-47b1-8317-faac871a2aa1'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:52.752074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac98be80d9d74f06b1596acfbe4bfcaf', 'request_id': 'req-8a9b6f99-b342-47b1-8317-faac871a2aa1'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:52.808097 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a39c22c6-ffad-48bb-9be3-aefd6985e29d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '043aaf5dfc4d4d0abc03e6566879f9f3'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:52.808097 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a39c22c6-ffad-48bb-9be3-aefd6985e29d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '043aaf5dfc4d4d0abc03e6566879f9f3'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:52.835042 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 763/1520] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:52 2022] DELETE /identity/v3/users/043aaf5dfc4d4d0abc03e6566879f9f3 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:52.840111 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29be56-3a6d-4b5f-94c6-e80f8828d4bf None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:52.840407 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29be56-3a6d-4b5f-94c6-e80f8828d4bf None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:52.841226 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29be56-3a6d-4b5f-94c6-e80f8828d4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:52.865179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-6f29be56-3a6d-4b5f-94c6-e80f8828d4bf None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:52.867134 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6f29be56-3a6d-4b5f-94c6-e80f8828d4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:52.892124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac98be80d9d74f06b1596acfbe4bfcaf', 'request_id': 'req-8a9b6f99-b342-47b1-8317-faac871a2aa1'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:52.913277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] The token cache is being invalidate because project ac98be80d9d74f06b1596acfbe4bfcaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:52.913933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:52.915644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:52.922191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 758/1521] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:52 2022] DELETE /identity/v3/projects/ac98be80d9d74f06b1596acfbe4bfcaf => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:52.926486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:52.926486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:53.044952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 764/1522] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:53.082301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b6f99-b342-47b1-8317-faac871a2aa1 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:53.095735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cb54e-f94c-4680-a344-7d2586da69da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:53.096556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb54e-f94c-4680-a344-7d2586da69da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:53.096849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb54e-f94c-4680-a344-7d2586da69da admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:53.097125 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cb54e-f94c-4680-a344-7d2586da69da admin admin] PATH_INFO: `/v3/users/4076ab33f5b64b9b80c34e5406c9fb91` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:53.098397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cb54e-f94c-4680-a344-7d2586da69da admin admin] RBAC: Authorizing `identity:delete_user(user_id=4076ab33f5b64b9b80c34e5406c9fb91)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:53.118248 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cb54e-f94c-4680-a344-7d2586da69da admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:53.122561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29be56-3a6d-4b5f-94c6-e80f8828d4bf None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:53.125066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6f29be56-3a6d-4b5f-94c6-e80f8828d4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:53.228157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29be56-3a6d-4b5f-94c6-e80f8828d4bf None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:53.232534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f6abb-38de-437e-8e16-fe1295284fa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:53.242668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f6abb-38de-437e-8e16-fe1295284fa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:53.242668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f6abb-38de-437e-8e16-fe1295284fa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:53.242668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f6abb-38de-437e-8e16-fe1295284fa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:53.251768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f6abb-38de-437e-8e16-fe1295284fa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:53.253548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f6abb-38de-437e-8e16-fe1295284fa9 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:53.272004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 765/1523] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:53.437217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-273cb54e-f94c-4680-a344-7d2586da69da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4076ab33f5b64b9b80c34e5406c9fb91'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:53.438685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-273cb54e-f94c-4680-a344-7d2586da69da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4076ab33f5b64b9b80c34e5406c9fb91'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:53.517863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-273cb54e-f94c-4680-a344-7d2586da69da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4076ab33f5b64b9b80c34e5406c9fb91'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:53.520613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-273cb54e-f94c-4680-a344-7d2586da69da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4076ab33f5b64b9b80c34e5406c9fb91'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:53.540431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-273cb54e-f94c-4680-a344-7d2586da69da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4076ab33f5b64b9b80c34e5406c9fb91'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:53.590347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 759/1524] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:52 2022] DELETE /identity/v3/users/4076ab33f5b64b9b80c34e5406c9fb91 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:54.199093 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:55.126276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f6abb-38de-437e-8e16-fe1295284fa9 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:55.128847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5f8f6abb-38de-437e-8e16-fe1295284fa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:55.158082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f6abb-38de-437e-8e16-fe1295284fa9 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:55.187559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:55.239127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:55.244040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:55.244040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:55.244040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] PATH_INFO: `/v3/projects/bd2df50848dd440fb1fa9752a01dde24` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:55.244040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd2df50848dd440fb1fa9752a01dde24)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:55.244040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:55.364000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dd6e9-21cc-475c-bc6d-4614481ddc52 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:55.364281 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dd6e9-21cc-475c-bc6d-4614481ddc52 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:55.364730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dd6e9-21cc-475c-bc6d-4614481ddc52 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:55.366132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 760/1525] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:32:55.388176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aaecff-1303-4083-8361-6da9702f0a29 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:55.388699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aaecff-1303-4083-8361-6da9702f0a29 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:55.389113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aaecff-1303-4083-8361-6da9702f0a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:55.395937 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:55.428699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd2df50848dd440fb1fa9752a01dde24', 'request_id': 'req-873fbc45-07ca-4c4f-bfc6-c70910a9dece'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.444140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd2df50848dd440fb1fa9752a01dde24', 'request_id': 'req-873fbc45-07ca-4c4f-bfc6-c70910a9dece'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.444246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c3aaecff-1303-4083-8361-6da9702f0a29 None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:55.444246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c3aaecff-1303-4083-8361-6da9702f0a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:55.507170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 761/1526] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:32:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:55.508022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aaecff-1303-4083-8361-6da9702f0a29 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:55.509342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c3aaecff-1303-4083-8361-6da9702f0a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:55.547129 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd2df50848dd440fb1fa9752a01dde24', 'request_id': 'req-873fbc45-07ca-4c4f-bfc6-c70910a9dece'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.558613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] The token cache is being invalidate because project bd2df50848dd440fb1fa9752a01dde24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:55.558836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.559981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.561814 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 766/1527] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:55 2022] DELETE /identity/v3/projects/bd2df50848dd440fb1fa9752a01dde24 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:55.562670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:55.563948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:55.574952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aaecff-1303-4083-8361-6da9702f0a29 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:55.580090 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fbc45-07ca-4c4f-bfc6-c70910a9dece admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:55.580291 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:55.580925 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:55.581208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:55.581485 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] PATH_INFO: `/v3/projects/49e64ba84d66400ea671b835030c7abf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:55.582641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=49e64ba84d66400ea671b835030c7abf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:55.584604 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:55.585768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7472d-4e39-443c-a71f-cf1cb13d149f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:55.586409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7472d-4e39-443c-a71f-cf1cb13d149f service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:55.591719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7472d-4e39-443c-a71f-cf1cb13d149f service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:55.591719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7472d-4e39-443c-a71f-cf1cb13d149f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:55.591719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9ec7472d-4e39-443c-a71f-cf1cb13d149f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:55.602166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7472d-4e39-443c-a71f-cf1cb13d149f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:55.603892 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7472d-4e39-443c-a71f-cf1cb13d149f service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:55.616238 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 767/1528] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:55.617145 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7472d-4e39-443c-a71f-cf1cb13d149f service nova] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:55.618442 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9ec7472d-4e39-443c-a71f-cf1cb13d149f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:55.632467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7472d-4e39-443c-a71f-cf1cb13d149f service nova] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:55.637158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:55.638353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:55.638898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:55.639007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] PATH_INFO: `/v3/users/8a3650c2cae941428bb00d916f1c9011` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:55.640155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a3650c2cae941428bb00d916f1c9011)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:55.642063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:55.742987 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49e64ba84d66400ea671b835030c7abf', 'request_id': 'req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.749714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49e64ba84d66400ea671b835030c7abf', 'request_id': 'req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.895813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49e64ba84d66400ea671b835030c7abf', 'request_id': 'req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.913129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] The token cache is being invalidate because project 49e64ba84d66400ea671b835030c7abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:55.913490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.913615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a3650c2cae941428bb00d916f1c9011'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.914456 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a3650c2cae941428bb00d916f1c9011'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.914578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-5ffdb9ce-6362-4877-b550-cc76f46ff4ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.920625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 762/1529] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:55 2022] DELETE /identity/v3/projects/49e64ba84d66400ea671b835030c7abf => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:55.921827 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993654d-8661-4da0-9b8e-c67316249fde None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:55.922136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993654d-8661-4da0-9b8e-c67316249fde None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:55.922853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993654d-8661-4da0-9b8e-c67316249fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:55.927540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a3650c2cae941428bb00d916f1c9011'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:55.945461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c993654d-8661-4da0-9b8e-c67316249fde None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:55.946973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c993654d-8661-4da0-9b8e-c67316249fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:56.019212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 763/1530] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:56.026583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f73b8-6353-408c-b3ed-9b9283142045 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.026873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f73b8-6353-408c-b3ed-9b9283142045 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.027153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f73b8-6353-408c-b3ed-9b9283142045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.047033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-441f73b8-6353-408c-b3ed-9b9283142045 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:56.048222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-441f73b8-6353-408c-b3ed-9b9283142045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:56.055544 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a3650c2cae941428bb00d916f1c9011'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:56.056221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a3650c2cae941428bb00d916f1c9011'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:56.056310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 764/1531] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:56.062388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f73b8-6353-408c-b3ed-9b9283142045 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:56.071682 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f73b8-6353-408c-b3ed-9b9283142045 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:56.076010 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef00aba-5070-4cf4-91a1-ade82909248e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:56.076542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 768/1532] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:55 2022] DELETE /identity/v3/users/8a3650c2cae941428bb00d916f1c9011 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:56.079635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef00aba-5070-4cf4-91a1-ade82909248e admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.079635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef00aba-5070-4cf4-91a1-ade82909248e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.079635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef00aba-5070-4cf4-91a1-ade82909248e admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.079635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef00aba-5070-4cf4-91a1-ade82909248e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:56.079635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef00aba-5070-4cf4-91a1-ade82909248e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:56.079973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef00aba-5070-4cf4-91a1-ade82909248e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:56.091539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 765/1533] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:32:56.095895 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:56.099582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:56.165292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affb843-eb53-4c62-aaaa-8b0b3ef06c0e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:56.170666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2f8d1-da43-4dc8-a918-2694ef59c49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:56.171278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2f8d1-da43-4dc8-a918-2694ef59c49f admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.171552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2f8d1-da43-4dc8-a918-2694ef59c49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.171846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2f8d1-da43-4dc8-a918-2694ef59c49f admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.172472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2f8d1-da43-4dc8-a918-2694ef59c49f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:56.174519 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2f8d1-da43-4dc8-a918-2694ef59c49f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:56.276175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 769/1534] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:56 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:56.280782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef00aba-5070-4cf4-91a1-ade82909248e admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:56.291107 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef00aba-5070-4cf4-91a1-ade82909248e admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:56.355870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21a194-83c5-4065-a670-d63e8b2eb6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:56.358451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21a194-83c5-4065-a670-d63e8b2eb6c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.358451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21a194-83c5-4065-a670-d63e8b2eb6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.358451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e21a194-83c5-4065-a670-d63e8b2eb6c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.358451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21a194-83c5-4065-a670-d63e8b2eb6c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:56.363317 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e21a194-83c5-4065-a670-d63e8b2eb6c1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:56.377073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:56.478674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 766/1535] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:56 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:56.483421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2f8d1-da43-4dc8-a918-2694ef59c49f admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:56.493331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2f8d1-da43-4dc8-a918-2694ef59c49f admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:56.497763 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9299723-de52-471f-8671-416aa8aee574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:56.498394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299723-de52-471f-8671-416aa8aee574 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.498671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299723-de52-471f-8671-416aa8aee574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.498942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9299723-de52-471f-8671-416aa8aee574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.499578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9299723-de52-471f-8671-416aa8aee574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:56.501420 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9299723-de52-471f-8671-416aa8aee574 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:56.511755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 770/1536] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:56.519107 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21a194-83c5-4065-a670-d63e8b2eb6c1 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:56.520697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9299723-de52-471f-8671-416aa8aee574 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:56.520968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3e21a194-83c5-4065-a670-d63e8b2eb6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:56.533806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9299723-de52-471f-8671-416aa8aee574 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:56.534359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e21a194-83c5-4065-a670-d63e8b2eb6c1 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:56.538755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe370b-9a3c-4b0f-83fb-d4adda0b5ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:56.546760 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe370b-9a3c-4b0f-83fb-d4adda0b5ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.547203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:56.547977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe370b-9a3c-4b0f-83fb-d4adda0b5ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.547977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe370b-9a3c-4b0f-83fb-d4adda0b5ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.548416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.548416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.548416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] PATH_INFO: `/v3/projects/63c0a11ce0544fc481b92e2462f833f8` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.548707 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe370b-9a3c-4b0f-83fb-d4adda0b5ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:56.548831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63c0a11ce0544fc481b92e2462f833f8)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:56.548961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe370b-9a3c-4b0f-83fb-d4adda0b5ac2 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:56.549048 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:56.554085 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 771/1537] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:32:56.558147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe370b-9a3c-4b0f-83fb-d4adda0b5ac2 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:56.567703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe370b-9a3c-4b0f-83fb-d4adda0b5ac2 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:56.572688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d05f7a-fd8e-40d1-a373-01024a469b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:56.573334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d05f7a-fd8e-40d1-a373-01024a469b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.575158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d05f7a-fd8e-40d1-a373-01024a469b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.575158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d05f7a-fd8e-40d1-a373-01024a469b6b admin admin] PATH_INFO: `/v3/projects/2e1a1430bfdb43d8bfffcfc3ee593c4c/users/4b93f882d8f6450ba17b681d62d9541f/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.587995 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d05f7a-fd8e-40d1-a373-01024a469b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1a1430bfdb43d8bfffcfc3ee593c4c, user_id=4b93f882d8f6450ba17b681d62d9541f, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:56.591181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d05f7a-fd8e-40d1-a373-01024a469b6b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:56.730426 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 772/1538] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:56 2022] PUT /identity/v3/projects/2e1a1430bfdb43d8bfffcfc3ee593c4c/users/4b93f882d8f6450ba17b681d62d9541f/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:56.740617 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63c0a11ce0544fc481b92e2462f833f8', 'request_id': 'req-6ebce71a-2d13-406f-a8da-a3745c461016'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:56.740781 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b7dfe-a7e0-4beb-a1f9-6f253f39a684 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.740956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b7dfe-a7e0-4beb-a1f9-6f253f39a684 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.741247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b7dfe-a7e0-4beb-a1f9-6f253f39a684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.748308 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63c0a11ce0544fc481b92e2462f833f8', 'request_id': 'req-6ebce71a-2d13-406f-a8da-a3745c461016'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:56.771500 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9d5b7dfe-a7e0-4beb-a1f9-6f253f39a684 None None] MFA Rules not processed for user `4b93f882d8f6450ba17b681d62d9541f`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:56.774787 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9d5b7dfe-a7e0-4beb-a1f9-6f253f39a684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:56.837114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 773/1539] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:56.843071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b7dfe-a7e0-4beb-a1f9-6f253f39a684 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:56.867131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63c0a11ce0544fc481b92e2462f833f8', 'request_id': 'req-6ebce71a-2d13-406f-a8da-a3745c461016'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:56.881517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] The token cache is being invalidate because project 63c0a11ce0544fc481b92e2462f833f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:32:56.882508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:56.883914 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6ebce71a-2d13-406f-a8da-a3745c461016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:56.889615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 767/1540] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:32:56 2022] DELETE /identity/v3/projects/63c0a11ce0544fc481b92e2462f833f8 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:56.891287 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7127d-676b-4975-a49a-84329b32abbe None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.891582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7127d-676b-4975-a49a-84329b32abbe None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.891847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7127d-676b-4975-a49a-84329b32abbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.910582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b7dfe-a7e0-4beb-a1f9-6f253f39a684 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:56.915734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-3cb7127d-676b-4975-a49a-84329b32abbe None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:56.916001 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04848b10-03a8-4baf-b530-fb955ec0cc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:56.916621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04848b10-03a8-4baf-b530-fb955ec0cc4c admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.916901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04848b10-03a8-4baf-b530-fb955ec0cc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.917176 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04848b10-03a8-4baf-b530-fb955ec0cc4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.917286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3cb7127d-676b-4975-a49a-84329b32abbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:56.917991 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04848b10-03a8-4baf-b530-fb955ec0cc4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:56.920094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04848b10-03a8-4baf-b530-fb955ec0cc4c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:56.932232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 768/1541] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:56.933748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7d2a6-f588-487f-a781-29d1c747245b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.934033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7d2a6-f588-487f-a781-29d1c747245b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.934971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7d2a6-f588-487f-a781-29d1c747245b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.954962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-39c7d2a6-f588-487f-a781-29d1c747245b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:56.956137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-39c7d2a6-f588-487f-a781-29d1c747245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:56.963795 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 769/1542] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:56.968490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7d2a6-f588-487f-a781-29d1c747245b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:56.978592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7d2a6-f588-487f-a781-29d1c747245b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:56.983068 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013a784-faaa-45bf-9b06-2f7de25068c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:56.984842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013a784-faaa-45bf-9b06-2f7de25068c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:56.984842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013a784-faaa-45bf-9b06-2f7de25068c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:56.984842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013a784-faaa-45bf-9b06-2f7de25068c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:56.984842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013a784-faaa-45bf-9b06-2f7de25068c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:32:56.985149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013a784-faaa-45bf-9b06-2f7de25068c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:56.986663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013a784-faaa-45bf-9b06-2f7de25068c9 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:56.998842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 770/1543] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:32:57.002953 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013a784-faaa-45bf-9b06-2f7de25068c9 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.012671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013a784-faaa-45bf-9b06-2f7de25068c9 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.024386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 774/1544] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:56 2022] POST /identity/v3/projects => generated 387 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:32:57.028529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fee98-61b9-4fce-8c95-1f7da74f5db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:57.029132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fee98-61b9-4fce-8c95-1f7da74f5db0 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.029407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fee98-61b9-4fce-8c95-1f7da74f5db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.029676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fee98-61b9-4fce-8c95-1f7da74f5db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.030411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fee98-61b9-4fce-8c95-1f7da74f5db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:57.030502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04848b10-03a8-4baf-b530-fb955ec0cc4c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.032271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fee98-61b9-4fce-8c95-1f7da74f5db0 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:57.042444 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-04848b10-03a8-4baf-b530-fb955ec0cc4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:57.123274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04848b10-03a8-4baf-b530-fb955ec0cc4c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.129921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07aa3b-3ee7-4d00-b5c6-aa9a7c22f268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:57.133384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07aa3b-3ee7-4d00-b5c6-aa9a7c22f268 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.133518 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 771/1545] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:57 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:32:57.133609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07aa3b-3ee7-4d00-b5c6-aa9a7c22f268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.133609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07aa3b-3ee7-4d00-b5c6-aa9a7c22f268 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.133609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07aa3b-3ee7-4d00-b5c6-aa9a7c22f268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:57.133806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07aa3b-3ee7-4d00-b5c6-aa9a7c22f268 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:57.134879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fee98-61b9-4fce-8c95-1f7da74f5db0 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.196334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fee98-61b9-4fce-8c95-1f7da74f5db0 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.201011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2f9d8-441e-4b59-940b-ba912a138fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:57.201634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2f9d8-441e-4b59-940b-ba912a138fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.202108 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2f9d8-441e-4b59-940b-ba912a138fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.202432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2f9d8-441e-4b59-940b-ba912a138fa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.203176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2f9d8-441e-4b59-940b-ba912a138fa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:57.205075 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2f9d8-441e-4b59-940b-ba912a138fa4 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:57.261814 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 775/1546] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:57 2022] POST /identity/v3/users => generated 300 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:57.265714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07aa3b-3ee7-4d00-b5c6-aa9a7c22f268 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.275157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07aa3b-3ee7-4d00-b5c6-aa9a7c22f268 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.280193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea88c8-fd75-4ba8-bc70-6a5f3a9c376a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:57.280769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea88c8-fd75-4ba8-bc70-6a5f3a9c376a admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.281048 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea88c8-fd75-4ba8-bc70-6a5f3a9c376a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.281318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea88c8-fd75-4ba8-bc70-6a5f3a9c376a admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.281942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea88c8-fd75-4ba8-bc70-6a5f3a9c376a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:57.283592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea88c8-fd75-4ba8-bc70-6a5f3a9c376a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:57.293134 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 776/1547] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:32:57.297668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea88c8-fd75-4ba8-bc70-6a5f3a9c376a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.306882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea88c8-fd75-4ba8-bc70-6a5f3a9c376a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.311475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db220c7b-84e4-44f2-ad89-78a255daa28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:57.312078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db220c7b-84e4-44f2-ad89-78a255daa28a admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.312352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db220c7b-84e4-44f2-ad89-78a255daa28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.312622 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db220c7b-84e4-44f2-ad89-78a255daa28a admin admin] PATH_INFO: `/v3/projects/cfee9b3b2d06493484852c234af80c2e/users/748af8d598374c2b8bea6414b77892db/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.327883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db220c7b-84e4-44f2-ad89-78a255daa28a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfee9b3b2d06493484852c234af80c2e, user_id=748af8d598374c2b8bea6414b77892db, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:57.330731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db220c7b-84e4-44f2-ad89-78a255daa28a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:57.367899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 772/1548] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:57 2022] POST /identity/v3/users => generated 283 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:32:57.369976 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2f9d8-441e-4b59-940b-ba912a138fa4 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.380581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2f9d8-441e-4b59-940b-ba912a138fa4 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.396585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9456d-249b-4422-8e28-9caf35376c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:57.397436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9456d-249b-4422-8e28-9caf35376c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.397708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9456d-249b-4422-8e28-9caf35376c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.397976 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9456d-249b-4422-8e28-9caf35376c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.398716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9456d-249b-4422-8e28-9caf35376c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:57.401914 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9456d-249b-4422-8e28-9caf35376c6a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:57.418369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 773/1549] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:32:57.418369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9456d-249b-4422-8e28-9caf35376c6a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.432286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9456d-249b-4422-8e28-9caf35376c6a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.446602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1889c34-91bd-4939-9b68-6efb0db6af3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:57.447239 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1889c34-91bd-4939-9b68-6efb0db6af3b admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.447522 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1889c34-91bd-4939-9b68-6efb0db6af3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.455094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1889c34-91bd-4939-9b68-6efb0db6af3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.455094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1889c34-91bd-4939-9b68-6efb0db6af3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:57.455094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1889c34-91bd-4939-9b68-6efb0db6af3b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:57.465656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 774/1550] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:57.469797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1889c34-91bd-4939-9b68-6efb0db6af3b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.488996 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1889c34-91bd-4939-9b68-6efb0db6af3b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.492850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 777/1551] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:57 2022] PUT /identity/v3/projects/cfee9b3b2d06493484852c234af80c2e/users/748af8d598374c2b8bea6414b77892db/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:57.496850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db220c7b-84e4-44f2-ad89-78a255daa28a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.582758 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53d801-804b-4a6c-99ba-fd991ad1df8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:57.583778 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53d801-804b-4a6c-99ba-fd991ad1df8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.584552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53d801-804b-4a6c-99ba-fd991ad1df8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.584659 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53d801-804b-4a6c-99ba-fd991ad1df8e admin admin] PATH_INFO: `/v3/projects/fd02c7532ef24674979cb3cbd2194bff/users/2e9d94307a01406db599522516094e56/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.599974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db220c7b-84e4-44f2-ad89-78a255daa28a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.607320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b3006-45c1-455a-b3d4-d0696dd8617e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:57.608040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3006-45c1-455a-b3d4-d0696dd8617e admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.608392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3006-45c1-455a-b3d4-d0696dd8617e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.608669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b3006-45c1-455a-b3d4-d0696dd8617e admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.609400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b3006-45c1-455a-b3d4-d0696dd8617e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:57.611277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b3006-45c1-455a-b3d4-d0696dd8617e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:57.612429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53d801-804b-4a6c-99ba-fd991ad1df8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd02c7532ef24674979cb3cbd2194bff, user_id=2e9d94307a01406db599522516094e56, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:57.618142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f53d801-804b-4a6c-99ba-fd991ad1df8e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:57.637493 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 778/1552] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:32:57.642552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b3006-45c1-455a-b3d4-d0696dd8617e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.663133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b3006-45c1-455a-b3d4-d0696dd8617e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.674015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0755d0-03ed-4df4-befc-5b0ae1b8a992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:57.674807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0755d0-03ed-4df4-befc-5b0ae1b8a992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.678699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0755d0-03ed-4df4-befc-5b0ae1b8a992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.678699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0755d0-03ed-4df4-befc-5b0ae1b8a992 admin admin] PATH_INFO: `/v3/projects/cfee9b3b2d06493484852c234af80c2e/users/748af8d598374c2b8bea6414b77892db/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.678699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0755d0-03ed-4df4-befc-5b0ae1b8a992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfee9b3b2d06493484852c234af80c2e, user_id=748af8d598374c2b8bea6414b77892db, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:57.680236 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0755d0-03ed-4df4-befc-5b0ae1b8a992 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:57.692042 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 779/1553] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:57 2022] PUT /identity/v3/projects/cfee9b3b2d06493484852c234af80c2e/users/748af8d598374c2b8bea6414b77892db/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:57.704688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544050c-b453-4870-a969-485dacb77dcd None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.705676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544050c-b453-4870-a969-485dacb77dcd None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.710779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544050c-b453-4870-a969-485dacb77dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.712361 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 775/1554] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:57 2022] PUT /identity/v3/projects/fd02c7532ef24674979cb3cbd2194bff/users/2e9d94307a01406db599522516094e56/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:57.724476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a02b1-dd51-4cc1-a2b5-c3e00eefb2d0 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.724811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a02b1-dd51-4cc1-a2b5-c3e00eefb2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.725144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a02b1-dd51-4cc1-a2b5-c3e00eefb2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.744797 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-4544050c-b453-4870-a969-485dacb77dcd None None] MFA Rules not processed for user `748af8d598374c2b8bea6414b77892db`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:57.746116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4544050c-b453-4870-a969-485dacb77dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:57.758708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-1c1a02b1-dd51-4cc1-a2b5-c3e00eefb2d0 None None] MFA Rules not processed for user `2e9d94307a01406db599522516094e56`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:57.759977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1c1a02b1-dd51-4cc1-a2b5-c3e00eefb2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:57.837064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 780/1555] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:32:57.849566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544050c-b453-4870-a969-485dacb77dcd None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.851617 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4544050c-b453-4870-a969-485dacb77dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:57.886758 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 776/1556] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:57.887917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a02b1-dd51-4cc1-a2b5-c3e00eefb2d0 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:57.948114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544050c-b453-4870-a969-485dacb77dcd None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.953030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39396cfe-7a6a-42c4-a941-1ade955c56d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:57.953982 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39396cfe-7a6a-42c4-a941-1ade955c56d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:57.954544 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39396cfe-7a6a-42c4-a941-1ade955c56d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:57.955027 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39396cfe-7a6a-42c4-a941-1ade955c56d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:57.970649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39396cfe-7a6a-42c4-a941-1ade955c56d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:57.972750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39396cfe-7a6a-42c4-a941-1ade955c56d7 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:57.994798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a02b1-dd51-4cc1-a2b5-c3e00eefb2d0 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:57.999779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 781/1557] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:32:58.001133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e190a01-cbdc-4594-88b2-25e93b56f8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:58.001870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e190a01-cbdc-4594-88b2-25e93b56f8e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:58.002181 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e190a01-cbdc-4594-88b2-25e93b56f8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:58.002457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e190a01-cbdc-4594-88b2-25e93b56f8e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:58.003192 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e190a01-cbdc-4594-88b2-25e93b56f8e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:58.005831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e190a01-cbdc-4594-88b2-25e93b56f8e5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:58.179552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 777/1558] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:32:57 2022] POST /identity/v3/projects => generated 365 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:58.187985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39396cfe-7a6a-42c4-a941-1ade955c56d7 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:58.205329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e190a01-cbdc-4594-88b2-25e93b56f8e5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:58.213916 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39396cfe-7a6a-42c4-a941-1ade955c56d7 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:58.368122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e190a01-cbdc-4594-88b2-25e93b56f8e5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:58.370542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95a395-4418-41e0-a803-25665998ed0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:58.371170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95a395-4418-41e0-a803-25665998ed0e admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:58.371459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95a395-4418-41e0-a803-25665998ed0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:58.371721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95a395-4418-41e0-a803-25665998ed0e admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:58.372625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95a395-4418-41e0-a803-25665998ed0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:58.381528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95a395-4418-41e0-a803-25665998ed0e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:58.408012 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5e0cf-8392-47a6-af80-2a273106657a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:58.408673 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5e0cf-8392-47a6-af80-2a273106657a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:58.408956 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5e0cf-8392-47a6-af80-2a273106657a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:58.409332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5e0cf-8392-47a6-af80-2a273106657a admin admin] PATH_INFO: `/v3/users/4b93f882d8f6450ba17b681d62d9541f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:58.412343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5e0cf-8392-47a6-af80-2a273106657a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b93f882d8f6450ba17b681d62d9541f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:58.414509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5e0cf-8392-47a6-af80-2a273106657a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:58.512686 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 782/1559] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:32:58 2022] POST /identity/v3/users => generated 289 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:32:58.514034 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95a395-4418-41e0-a803-25665998ed0e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:58.524558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95a395-4418-41e0-a803-25665998ed0e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:58.529765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430b82-1e54-43a5-9fc9-d5e09c252538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:58.530472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430b82-1e54-43a5-9fc9-d5e09c252538 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:58.530882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430b82-1e54-43a5-9fc9-d5e09c252538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:58.531169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430b82-1e54-43a5-9fc9-d5e09c252538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:58.531821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14430b82-1e54-43a5-9fc9-d5e09c252538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:58.533427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14430b82-1e54-43a5-9fc9-d5e09c252538 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:58.546078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 783/1560] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:32:58.556030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430b82-1e54-43a5-9fc9-d5e09c252538 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:58.566799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430b82-1e54-43a5-9fc9-d5e09c252538 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:58.572005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ceeaf-735b-4068-971b-4b214a25ce6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:58.582958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ceeaf-735b-4068-971b-4b214a25ce6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:58.582958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ceeaf-735b-4068-971b-4b214a25ce6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:58.582958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ceeaf-735b-4068-971b-4b214a25ce6f admin admin] PATH_INFO: `/v3/projects/32bb9868f33647678ecf8eca89d7ba42/users/dacc58cbc3614a1c9e1caed5af6efd5e/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:58.603445 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ceeaf-735b-4068-971b-4b214a25ce6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=32bb9868f33647678ecf8eca89d7ba42, user_id=dacc58cbc3614a1c9e1caed5af6efd5e, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:58.606303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ceeaf-735b-4068-971b-4b214a25ce6f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:58.705014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-08f5e0cf-8392-47a6-af80-2a273106657a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b93f882d8f6450ba17b681d62d9541f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:58.706094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-08f5e0cf-8392-47a6-af80-2a273106657a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b93f882d8f6450ba17b681d62d9541f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:58.727847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 784/1561] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:58 2022] PUT /identity/v3/projects/32bb9868f33647678ecf8eca89d7ba42/users/dacc58cbc3614a1c9e1caed5af6efd5e/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:58.731873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ceeaf-735b-4068-971b-4b214a25ce6f admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:58.733449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ff1ceeaf-735b-4068-971b-4b214a25ce6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:58.804670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-08f5e0cf-8392-47a6-af80-2a273106657a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b93f882d8f6450ba17b681d62d9541f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:58.805394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-08f5e0cf-8392-47a6-af80-2a273106657a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b93f882d8f6450ba17b681d62d9541f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:58.816668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-08f5e0cf-8392-47a6-af80-2a273106657a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b93f882d8f6450ba17b681d62d9541f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:58.822621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ceeaf-735b-4068-971b-4b214a25ce6f admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:58.829117 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13df68-bff9-4a6a-b660-f32c5675f323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:58.829731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13df68-bff9-4a6a-b660-f32c5675f323 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:58.830011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13df68-bff9-4a6a-b660-f32c5675f323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:58.830311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13df68-bff9-4a6a-b660-f32c5675f323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:58.831473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13df68-bff9-4a6a-b660-f32c5675f323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:58.832663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13df68-bff9-4a6a-b660-f32c5675f323 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:58.845451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 785/1562] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:32:58.849671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13df68-bff9-4a6a-b660-f32c5675f323 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:58.859221 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 778/1563] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:58 2022] DELETE /identity/v3/users/4b93f882d8f6450ba17b681d62d9541f => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:32:58.860745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601d287-3933-44f9-bd2a-b5e03c1296ac None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:58.861235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601d287-3933-44f9-bd2a-b5e03c1296ac None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:58.861608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601d287-3933-44f9-bd2a-b5e03c1296ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:58.863924 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13df68-bff9-4a6a-b660-f32c5675f323 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:58.902767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8601d287-3933-44f9-bd2a-b5e03c1296ac None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:58.906893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8601d287-3933-44f9-bd2a-b5e03c1296ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:58.970306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:58.977330 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279660ab-4901-4764-bcfd-937e670854e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:58.978239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279660ab-4901-4764-bcfd-937e670854e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:58.978682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279660ab-4901-4764-bcfd-937e670854e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:58.979095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279660ab-4901-4764-bcfd-937e670854e8 admin admin] PATH_INFO: `/v3/projects/32bb9868f33647678ecf8eca89d7ba42/users/dacc58cbc3614a1c9e1caed5af6efd5e/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:58.981549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279660ab-4901-4764-bcfd-937e670854e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32bb9868f33647678ecf8eca89d7ba42, user_id=dacc58cbc3614a1c9e1caed5af6efd5e, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:59.003786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279660ab-4901-4764-bcfd-937e670854e8 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:59.021210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 779/1564] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:59.025242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 786/1565] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:32:58 2022] PUT /identity/v3/projects/32bb9868f33647678ecf8eca89d7ba42/users/dacc58cbc3614a1c9e1caed5af6efd5e/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:32:59.037597 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d6e30-b773-4c94-b552-bae18e7be138 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:59.037887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d6e30-b773-4c94-b552-bae18e7be138 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:59.038197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d6e30-b773-4c94-b552-bae18e7be138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:59.042805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279660ab-4901-4764-bcfd-937e670854e8 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:59.047090 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-279660ab-4901-4764-bcfd-937e670854e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:59.062800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:32:59.082529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-581d6e30-b773-4c94-b552-bae18e7be138 None None] MFA Rules not processed for user `dacc58cbc3614a1c9e1caed5af6efd5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:32:59.084053 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-581d6e30-b773-4c94-b552-bae18e7be138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:59.176464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279660ab-4901-4764-bcfd-937e670854e8 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:59.180706 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23255401-2384-42c0-ae86-6d6ce00462c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:59.181511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23255401-2384-42c0-ae86-6d6ce00462c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:59.181978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23255401-2384-42c0-ae86-6d6ce00462c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:59.182291 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23255401-2384-42c0-ae86-6d6ce00462c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:59.217730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 780/1566] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:32:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:32:59.224760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d6e30-b773-4c94-b552-bae18e7be138 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:59.226204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-581d6e30-b773-4c94-b552-bae18e7be138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:59.307619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d6e30-b773-4c94-b552-bae18e7be138 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:59.313938 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1b4a5-80ab-4513-88fb-4f936f32f0a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:59.314582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1b4a5-80ab-4513-88fb-4f936f32f0a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:59.315657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1b4a5-80ab-4513-88fb-4f936f32f0a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:59.315951 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1b4a5-80ab-4513-88fb-4f936f32f0a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:59.317903 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23255401-2384-42c0-ae86-6d6ce00462c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:59.319472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23255401-2384-42c0-ae86-6d6ce00462c2 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:59.353056 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1b4a5-80ab-4513-88fb-4f936f32f0a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:59.366524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1b4a5-80ab-4513-88fb-4f936f32f0a0 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:59.370911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 787/1567] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:32:59.376759 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 781/1568] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:32:59.501480 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23255401-2384-42c0-ae86-6d6ce00462c2 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:59.503139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-23255401-2384-42c0-ae86-6d6ce00462c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:59.526473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23255401-2384-42c0-ae86-6d6ce00462c2 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:59.531685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4372cb7-bec5-439f-b2ac-4223937ebd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:59.532553 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4372cb7-bec5-439f-b2ac-4223937ebd8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:59.532986 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4372cb7-bec5-439f-b2ac-4223937ebd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:59.533434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4372cb7-bec5-439f-b2ac-4223937ebd8a admin admin] PATH_INFO: `/v3/users/5269b87577834012ad5d2336f115f78c` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:59.534773 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4372cb7-bec5-439f-b2ac-4223937ebd8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5269b87577834012ad5d2336f115f78c)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:59.545924 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4372cb7-bec5-439f-b2ac-4223937ebd8a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:32:59.846173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1b4a5-80ab-4513-88fb-4f936f32f0a0 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:32:59.858586 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1b4a5-80ab-4513-88fb-4f936f32f0a0 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:32:59.874525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705af210-a4cb-4045-ad39-94b8778526c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:32:59.875302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705af210-a4cb-4045-ad39-94b8778526c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:32:59.875580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705af210-a4cb-4045-ad39-94b8778526c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:32:59.875855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705af210-a4cb-4045-ad39-94b8778526c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:32:59.878358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-705af210-a4cb-4045-ad39-94b8778526c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:32:59.900109 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a4372cb7-bec5-439f-b2ac-4223937ebd8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5269b87577834012ad5d2336f115f78c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:59.904033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a4372cb7-bec5-439f-b2ac-4223937ebd8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5269b87577834012ad5d2336f115f78c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:59.920494 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a4372cb7-bec5-439f-b2ac-4223937ebd8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5269b87577834012ad5d2336f115f78c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:32:59.990354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705af210-a4cb-4045-ad39-94b8778526c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:32:59.991874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705af210-a4cb-4045-ad39-94b8778526c7 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:00.002582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 782/1569] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:32:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:33:00.005789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705af210-a4cb-4045-ad39-94b8778526c7 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:00.007259 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-705af210-a4cb-4045-ad39-94b8778526c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:00.022321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705af210-a4cb-4045-ad39-94b8778526c7 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:00.029561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:00.030263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:00.030520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:00.030807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] PATH_INFO: `/v3/projects/2e1a1430bfdb43d8bfffcfc3ee593c4c` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:00.031920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e1a1430bfdb43d8bfffcfc3ee593c4c)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:00.034074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:00.044695 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a4372cb7-bec5-439f-b2ac-4223937ebd8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5269b87577834012ad5d2336f115f78c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:00.045322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a4372cb7-bec5-439f-b2ac-4223937ebd8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5269b87577834012ad5d2336f115f78c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:00.086879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 788/1570] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:32:59 2022] DELETE /identity/v3/users/5269b87577834012ad5d2336f115f78c => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:00.092373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e95696-a3fe-49c9-ad53-0f3d8ab597a0 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:00.092698 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e95696-a3fe-49c9-ad53-0f3d8ab597a0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:00.093776 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e95696-a3fe-49c9-ad53-0f3d8ab597a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:00.145225 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-b2e95696-a3fe-49c9-ad53-0f3d8ab597a0 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:00.146587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b2e95696-a3fe-49c9-ad53-0f3d8ab597a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:00.216862 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e1a1430bfdb43d8bfffcfc3ee593c4c', 'request_id': 'req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:00.244874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e1a1430bfdb43d8bfffcfc3ee593c4c', 'request_id': 'req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:00.245749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:00.291863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 789/1571] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:00.302300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e95696-a3fe-49c9-ad53-0f3d8ab597a0 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:00.303880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b2e95696-a3fe-49c9-ad53-0f3d8ab597a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:00.359030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e1a1430bfdb43d8bfffcfc3ee593c4c', 'request_id': 'req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:00.370365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] The token cache is being invalidate because project 2e1a1430bfdb43d8bfffcfc3ee593c4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:00.372943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:00.372943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:00.373471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 783/1572] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:00 2022] DELETE /identity/v3/projects/2e1a1430bfdb43d8bfffcfc3ee593c4c => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:00.377976 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:00.379725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:00.397419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e95696-a3fe-49c9-ad53-0f3d8ab597a0 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:00.485616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77413a-74cd-4ac4-8da9-e04e3d64c46f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:00.487982 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77413a-74cd-4ac4-8da9-e04e3d64c46f service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:00.488273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77413a-74cd-4ac4-8da9-e04e3d64c46f service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:00.488597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77413a-74cd-4ac4-8da9-e04e3d64c46f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:00.496580 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ea77413a-74cd-4ac4-8da9-e04e3d64c46f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:00.514405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b08d4-cac9-4e1f-84bc-58ed4e22a13f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:00.519472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:00.520175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:00.520463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:00.520893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] PATH_INFO: `/v3/users/748af8d598374c2b8bea6414b77892db` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:00.522612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=748af8d598374c2b8bea6414b77892db)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:00.524662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:00.529693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77413a-74cd-4ac4-8da9-e04e3d64c46f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:00.532095 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77413a-74cd-4ac4-8da9-e04e3d64c46f service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:00.557874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 790/1573] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:00.920548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:00.954114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '748af8d598374c2b8bea6414b77892db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:00.955131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '748af8d598374c2b8bea6414b77892db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:01.308715 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '748af8d598374c2b8bea6414b77892db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:01.309900 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '748af8d598374c2b8bea6414b77892db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:01.319412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '748af8d598374c2b8bea6414b77892db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:01.358077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 784/1574] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:00 2022] DELETE /identity/v3/users/748af8d598374c2b8bea6414b77892db => generated 0 bytes in 975 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:01.396230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:02.062250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77413a-74cd-4ac4-8da9-e04e3d64c46f service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:02.069707 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ea77413a-74cd-4ac4-8da9-e04e3d64c46f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:02.076180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77413a-74cd-4ac4-8da9-e04e3d64c46f service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:02.191616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:02.192472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:02.192761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:02.193096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] PATH_INFO: `/v3/projects/7fc712d2b62445a8bc9a90516c970dee` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:02.194406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc712d2b62445a8bc9a90516c970dee)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:02.198086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:02.367033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc712d2b62445a8bc9a90516c970dee', 'request_id': 'req-90f369db-cb48-4459-aed4-c53d27112e75'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:02.379943 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc712d2b62445a8bc9a90516c970dee', 'request_id': 'req-90f369db-cb48-4459-aed4-c53d27112e75'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:02.514844 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc712d2b62445a8bc9a90516c970dee', 'request_id': 'req-90f369db-cb48-4459-aed4-c53d27112e75'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:02.530728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] The token cache is being invalidate because project 7fc712d2b62445a8bc9a90516c970dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:02.533448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:02.549031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:02.552004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 791/1575] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:02 2022] DELETE /identity/v3/projects/7fc712d2b62445a8bc9a90516c970dee => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:02.556255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:02.558917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:02.595234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76605f02-d331-46d0-8943-ac3db53dd234 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:02.691503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d60416-7977-4882-9189-355775776e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:02.693050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d60416-7977-4882-9189-355775776e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:02.693504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d60416-7977-4882-9189-355775776e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:02.693914 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d60416-7977-4882-9189-355775776e5d admin admin] PATH_INFO: `/v3/users/bf3263d2aab24aea819859f564291fc4` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:02.695438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d60416-7977-4882-9189-355775776e5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf3263d2aab24aea819859f564291fc4)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:02.697535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d60416-7977-4882-9189-355775776e5d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:02.731115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:03.054078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-43d60416-7977-4882-9189-355775776e5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf3263d2aab24aea819859f564291fc4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.055067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-43d60416-7977-4882-9189-355775776e5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf3263d2aab24aea819859f564291fc4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.147124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-43d60416-7977-4882-9189-355775776e5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf3263d2aab24aea819859f564291fc4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.150656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-43d60416-7977-4882-9189-355775776e5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf3263d2aab24aea819859f564291fc4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.156723 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-43d60416-7977-4882-9189-355775776e5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf3263d2aab24aea819859f564291fc4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.181554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 785/1576] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:02 2022] DELETE /identity/v3/users/bf3263d2aab24aea819859f564291fc4 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:03.226566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:03.228817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:03.243027 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f369db-cb48-4459-aed4-c53d27112e75 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:03.250866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d60416-7977-4882-9189-355775776e5d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:03.252109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-43d60416-7977-4882-9189-355775776e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:03.369678 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:03.370681 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:03.371187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:03.371697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] PATH_INFO: `/v3/projects/cfee9b3b2d06493484852c234af80c2e` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:03.373360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfee9b3b2d06493484852c234af80c2e)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:03.373475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d60416-7977-4882-9189-355775776e5d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:03.375671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:03.383566 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fa2d2-fbcb-470c-babd-aada855afdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:03.384225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fa2d2-fbcb-470c-babd-aada855afdf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:03.384502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fa2d2-fbcb-470c-babd-aada855afdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:03.384822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fa2d2-fbcb-470c-babd-aada855afdf6 admin admin] PATH_INFO: `/v3/users/988fa9f598bd4836a7ff88cee07c717f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:03.396250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fa2d2-fbcb-470c-babd-aada855afdf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=988fa9f598bd4836a7ff88cee07c717f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:03.405312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fa2d2-fbcb-470c-babd-aada855afdf6 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:03.697647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-553fa2d2-fbcb-470c-babd-aada855afdf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '988fa9f598bd4836a7ff88cee07c717f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.699377 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-553fa2d2-fbcb-470c-babd-aada855afdf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '988fa9f598bd4836a7ff88cee07c717f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.708781 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfee9b3b2d06493484852c234af80c2e', 'request_id': 'req-fafce866-3416-4235-a9e1-65d8802b812d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.720352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfee9b3b2d06493484852c234af80c2e', 'request_id': 'req-fafce866-3416-4235-a9e1-65d8802b812d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.812304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-553fa2d2-fbcb-470c-babd-aada855afdf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '988fa9f598bd4836a7ff88cee07c717f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.813640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-553fa2d2-fbcb-470c-babd-aada855afdf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '988fa9f598bd4836a7ff88cee07c717f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.834515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-553fa2d2-fbcb-470c-babd-aada855afdf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '988fa9f598bd4836a7ff88cee07c717f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.861707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 786/1577] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:03 2022] DELETE /identity/v3/users/988fa9f598bd4836a7ff88cee07c717f => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:03.867913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d15e4-ee76-4200-b94f-6b0262b99e67 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:03.868364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d15e4-ee76-4200-b94f-6b0262b99e67 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:03.868836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d15e4-ee76-4200-b94f-6b0262b99e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:03.900718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-074d15e4-ee76-4200-b94f-6b0262b99e67 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:03.902180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-074d15e4-ee76-4200-b94f-6b0262b99e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:03.902620 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:03.960010 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfee9b3b2d06493484852c234af80c2e', 'request_id': 'req-fafce866-3416-4235-a9e1-65d8802b812d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.972704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 787/1578] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:03.977158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] The token cache is being invalidate because project cfee9b3b2d06493484852c234af80c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:03.977727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.978731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:03.981015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 792/1579] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:03 2022] DELETE /identity/v3/projects/cfee9b3b2d06493484852c234af80c2e => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:03.988478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451e1be-4140-475a-a908-31689dcf0f2a None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:03.989114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451e1be-4140-475a-a908-31689dcf0f2a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:03.989551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451e1be-4140-475a-a908-31689dcf0f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:04.011021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:04.013749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:04.016610 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-3451e1be-4140-475a-a908-31689dcf0f2a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:04.018933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3451e1be-4140-475a-a908-31689dcf0f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:04.034184 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 788/1580] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:04.040359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246f1fb-769e-4b3e-b079-2fc4707a4f28 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:04.040904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246f1fb-769e-4b3e-b079-2fc4707a4f28 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:04.041187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246f1fb-769e-4b3e-b079-2fc4707a4f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:04.079788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b246f1fb-769e-4b3e-b079-2fc4707a4f28 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:04.079788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b246f1fb-769e-4b3e-b079-2fc4707a4f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:04.090111 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 789/1581] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:04.091999 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246f1fb-769e-4b3e-b079-2fc4707a4f28 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:04.103425 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246f1fb-769e-4b3e-b079-2fc4707a4f28 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:04.109754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57e671-d927-402f-8a24-12ee7d3a4c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:04.110698 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57e671-d927-402f-8a24-12ee7d3a4c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:04.111240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57e671-d927-402f-8a24-12ee7d3a4c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:04.111707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57e671-d927-402f-8a24-12ee7d3a4c4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:04.112561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57e671-d927-402f-8a24-12ee7d3a4c4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:33:04.113350 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57e671-d927-402f-8a24-12ee7d3a4c4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:04.115965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57e671-d927-402f-8a24-12ee7d3a4c4e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:04.134842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 790/1582] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:33:04.136823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57e671-d927-402f-8a24-12ee7d3a4c4e admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:04.138578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafce866-3416-4235-a9e1-65d8802b812d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:04.149351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57e671-d927-402f-8a24-12ee7d3a4c4e admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:04.155763 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac8971-3468-45e8-89fd-ed6390b94904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:04.156804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac8971-3468-45e8-89fd-ed6390b94904 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:04.157398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac8971-3468-45e8-89fd-ed6390b94904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:04.157859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac8971-3468-45e8-89fd-ed6390b94904 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:04.158730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac8971-3468-45e8-89fd-ed6390b94904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:04.160863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac8971-3468-45e8-89fd-ed6390b94904 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:04.210416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cc4a4-2683-4fc8-8101-790fda51449c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:04.211230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cc4a4-2683-4fc8-8101-790fda51449c service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:04.211579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cc4a4-2683-4fc8-8101-790fda51449c service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:04.211918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cc4a4-2683-4fc8-8101-790fda51449c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:04.214541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-341cc4a4-2683-4fc8-8101-790fda51449c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:04.228779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341cc4a4-2683-4fc8-8101-790fda51449c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:04.232166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341cc4a4-2683-4fc8-8101-790fda51449c service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:04.245215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 793/1583] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:04.308405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:04.317522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cc4a4-2683-4fc8-8101-790fda51449c service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:04.319681 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-341cc4a4-2683-4fc8-8101-790fda51449c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:04.326608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 791/1584] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:04 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:04.337941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac8971-3468-45e8-89fd-ed6390b94904 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:04.339580 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cc4a4-2683-4fc8-8101-790fda51449c service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:04.472736 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:04.473386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:04.473661 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:04.476472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] PATH_INFO: `/v3/projects/dba67cb1b0b9462c90f4ca01df620284` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:04.476472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba67cb1b0b9462c90f4ca01df620284)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:04.478592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:04.541040 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac8971-3468-45e8-89fd-ed6390b94904 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:04.559492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e7d49-89ad-42ae-987f-04b767186fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:04.560565 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e7d49-89ad-42ae-987f-04b767186fce admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:04.561140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e7d49-89ad-42ae-987f-04b767186fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:04.561419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e7d49-89ad-42ae-987f-04b767186fce admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:04.564366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e7d49-89ad-42ae-987f-04b767186fce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:04.567615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e7d49-89ad-42ae-987f-04b767186fce admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:04.648496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba67cb1b0b9462c90f4ca01df620284', 'request_id': 'req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:04.655548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba67cb1b0b9462c90f4ca01df620284', 'request_id': 'req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:04.830652 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba67cb1b0b9462c90f4ca01df620284', 'request_id': 'req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:04.838165 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 792/1585] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:04 2022] POST /identity/v3/users => generated 292 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:04.842630 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e7d49-89ad-42ae-987f-04b767186fce admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:04.855949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e7d49-89ad-42ae-987f-04b767186fce admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:04.861541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] The token cache is being invalidate because project dba67cb1b0b9462c90f4ca01df620284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:04.862059 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:04.862970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1f6ed5cd-dc37-48bf-8a60-c82f8e5f2a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:04.864925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 794/1586] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:04 2022] DELETE /identity/v3/projects/dba67cb1b0b9462c90f4ca01df620284 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:04.869764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b046-4baa-4dd7-a2a8-7d7e49329424 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:04.870585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b046-4baa-4dd7-a2a8-7d7e49329424 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:04.871015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585b046-4baa-4dd7-a2a8-7d7e49329424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:04.904557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7585b046-4baa-4dd7-a2a8-7d7e49329424 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:04.906978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7585b046-4baa-4dd7-a2a8-7d7e49329424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:04.951944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ac0c-19ac-4a0d-a404-3c5bda86f32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:04.952722 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ac0c-19ac-4a0d-a404-3c5bda86f32a admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:04.953081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ac0c-19ac-4a0d-a404-3c5bda86f32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:04.953589 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ac0c-19ac-4a0d-a404-3c5bda86f32a admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:04.954474 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619ac0c-19ac-4a0d-a404-3c5bda86f32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:04.956092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619ac0c-19ac-4a0d-a404-3c5bda86f32a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:04.967720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 793/1587] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:04.971740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ac0c-19ac-4a0d-a404-3c5bda86f32a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:04.973521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0619ac0c-19ac-4a0d-a404-3c5bda86f32a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:04.979235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 795/1588] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:04.987319 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ac0c-19ac-4a0d-a404-3c5bda86f32a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:04.989018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96c128-663d-423d-8e49-457d3c26d848 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:04.989304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96c128-663d-423d-8e49-457d3c26d848 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:04.989573 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96c128-663d-423d-8e49-457d3c26d848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:04.992650 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d4b11-156d-4852-80d9-b70de68ad31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:04.993957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d4b11-156d-4852-80d9-b70de68ad31a admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:04.994501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d4b11-156d-4852-80d9-b70de68ad31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:04.994952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d4b11-156d-4852-80d9-b70de68ad31a admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:04.995790 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d4b11-156d-4852-80d9-b70de68ad31a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:04.997694 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d4b11-156d-4852-80d9-b70de68ad31a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:05.015560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 794/1589] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:05.020225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d4b11-156d-4852-80d9-b70de68ad31a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:05.032193 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d4b11-156d-4852-80d9-b70de68ad31a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:05.043191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c1cf1-cd20-4950-86f5-b5ca30a98952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:05.044625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c1cf1-cd20-4950-86f5-b5ca30a98952 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:05.045083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c1cf1-cd20-4950-86f5-b5ca30a98952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:05.045470 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c1cf1-cd20-4950-86f5-b5ca30a98952 admin admin] PATH_INFO: `/v3/projects/bb0a1301ccd1479a864b58d704b8421b/users/ab0d96ea9de84797a7d3cb233799f850/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:05.065858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c1cf1-cd20-4950-86f5-b5ca30a98952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb0a1301ccd1479a864b58d704b8421b, user_id=ab0d96ea9de84797a7d3cb233799f850, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:05.069303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c1cf1-cd20-4950-86f5-b5ca30a98952 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:05.149686 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-da96c128-663d-423d-8e49-457d3c26d848 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:05.150957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-da96c128-663d-423d-8e49-457d3c26d848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:05.161548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 796/1590] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:05.175769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96c128-663d-423d-8e49-457d3c26d848 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:05.175906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 795/1591] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:05 2022] PUT /identity/v3/projects/bb0a1301ccd1479a864b58d704b8421b/users/ab0d96ea9de84797a7d3cb233799f850/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:05.177984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd29a09-b60e-4b54-841f-7fc65a74050e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:05.178300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd29a09-b60e-4b54-841f-7fc65a74050e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:05.178572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd29a09-b60e-4b54-841f-7fc65a74050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:05.181567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96c128-663d-423d-8e49-457d3c26d848 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:05.212754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-6fd29a09-b60e-4b54-841f-7fc65a74050e None None] MFA Rules not processed for user `ab0d96ea9de84797a7d3cb233799f850`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:05.213955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6fd29a09-b60e-4b54-841f-7fc65a74050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:05.304021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1988d95-469a-406c-81a6-34693e3e74aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:05.305218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1988d95-469a-406c-81a6-34693e3e74aa admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:05.305218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1988d95-469a-406c-81a6-34693e3e74aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:05.305218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1988d95-469a-406c-81a6-34693e3e74aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:05.306338 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1988d95-469a-406c-81a6-34693e3e74aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:33:05.306645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1988d95-469a-406c-81a6-34693e3e74aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:05.308246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1988d95-469a-406c-81a6-34693e3e74aa admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:05.351303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 797/1592] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:33:05.351303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1988d95-469a-406c-81a6-34693e3e74aa admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:05.352413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 796/1593] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:05.353433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd29a09-b60e-4b54-841f-7fc65a74050e None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:05.358518 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d1988d95-469a-406c-81a6-34693e3e74aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:05.385384 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd29a09-b60e-4b54-841f-7fc65a74050e None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:05.390548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1988d95-469a-406c-81a6-34693e3e74aa admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:05.400706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41744e-4d8f-45a4-93a9-aa1196086299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:05.401450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41744e-4d8f-45a4-93a9-aa1196086299 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:05.401732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41744e-4d8f-45a4-93a9-aa1196086299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:05.402864 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41744e-4d8f-45a4-93a9-aa1196086299 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:05.404366 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:05.405132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:05.405263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41744e-4d8f-45a4-93a9-aa1196086299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:05.406606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:05.407545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] PATH_INFO: `/v3/projects/3b20a380c6e6487e805f4d011a00a423` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:05.410187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b20a380c6e6487e805f4d011a00a423)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:05.415948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:05.418101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41744e-4d8f-45a4-93a9-aa1196086299 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:05.561304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 797/1594] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:05 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:05.563448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41744e-4d8f-45a4-93a9-aa1196086299 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:05.575835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b20a380c6e6487e805f4d011a00a423', 'request_id': 'req-003234e2-9cbb-49bb-8db8-59759f8b502a'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:05.586136 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b20a380c6e6487e805f4d011a00a423', 'request_id': 'req-003234e2-9cbb-49bb-8db8-59759f8b502a'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:05.669889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b20a380c6e6487e805f4d011a00a423', 'request_id': 'req-003234e2-9cbb-49bb-8db8-59759f8b502a'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:05.682846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41744e-4d8f-45a4-93a9-aa1196086299 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:05.708064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] The token cache is being invalidate because project 3b20a380c6e6487e805f4d011a00a423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:05.708363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:05.712717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:05.715272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 798/1595] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:05 2022] DELETE /identity/v3/projects/3b20a380c6e6487e805f4d011a00a423 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:05.719140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:05.745770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:05.832518 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4787d3-a04f-4dd5-b462-38df0cfaf1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:05.833416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4787d3-a04f-4dd5-b462-38df0cfaf1db admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:05.833702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4787d3-a04f-4dd5-b462-38df0cfaf1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:05.833967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4787d3-a04f-4dd5-b462-38df0cfaf1db admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:05.834797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4787d3-a04f-4dd5-b462-38df0cfaf1db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:05.836726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4787d3-a04f-4dd5-b462-38df0cfaf1db admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:05.885983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003234e2-9cbb-49bb-8db8-59759f8b502a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:05.899412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe015291-c9fd-4f00-bd55-3057447da650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:05.900322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe015291-c9fd-4f00-bd55-3057447da650 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:05.900710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe015291-c9fd-4f00-bd55-3057447da650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:05.901133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe015291-c9fd-4f00-bd55-3057447da650 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:05.901920 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe015291-c9fd-4f00-bd55-3057447da650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:05.910845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe015291-c9fd-4f00-bd55-3057447da650 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:06.025741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 798/1596] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:05 2022] POST /identity/v3/projects => generated 383 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:06.033482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4787d3-a04f-4dd5-b462-38df0cfaf1db admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:06.039196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cc4787d3-a04f-4dd5-b462-38df0cfaf1db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:06.046754 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 799/1597] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:05 2022] POST /identity/v3/users => generated 289 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:06.047782 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe015291-c9fd-4f00-bd55-3057447da650 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:06.052214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-fe015291-c9fd-4f00-bd55-3057447da650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:06.070061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe015291-c9fd-4f00-bd55-3057447da650 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:06.131435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:06.193050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4787d3-a04f-4dd5-b462-38df0cfaf1db admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:06.198516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:06.199179 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:06.199466 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:06.199757 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] PATH_INFO: `/v3/users/35689fe0ec754caf8be9a351f110cc3f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:06.201508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35689fe0ec754caf8be9a351f110cc3f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:06.203696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:06.209151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc9cf7-ba8f-4368-8c19-c4f9d9de82b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:06.211575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc9cf7-ba8f-4368-8c19-c4f9d9de82b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:06.211575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc9cf7-ba8f-4368-8c19-c4f9d9de82b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:06.211575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc9cf7-ba8f-4368-8c19-c4f9d9de82b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:06.212493 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc9cf7-ba8f-4368-8c19-c4f9d9de82b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:06.214910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc9cf7-ba8f-4368-8c19-c4f9d9de82b2 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:06.871652 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 800/1598] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:06 2022] POST /identity/v3/users => generated 298 bytes in 824 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:06.872975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc9cf7-ba8f-4368-8c19-c4f9d9de82b2 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:06.890604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc9cf7-ba8f-4368-8c19-c4f9d9de82b2 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:06.964433 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ac009-f389-46ba-ab49-44aae185c4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:06.964433 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ac009-f389-46ba-ab49-44aae185c4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:06.964433 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ac009-f389-46ba-ab49-44aae185c4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:06.964433 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ac009-f389-46ba-ab49-44aae185c4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:06.964761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ac009-f389-46ba-ab49-44aae185c4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:06.964761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ac009-f389-46ba-ab49-44aae185c4a5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:06.964761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 801/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:06.964761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ac009-f389-46ba-ab49-44aae185c4a5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:06.964761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ac009-f389-46ba-ab49-44aae185c4a5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:06.965108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad21e14-4d4b-4862-915e-eee1881c79d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:06.965108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad21e14-4d4b-4862-915e-eee1881c79d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:06.965108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad21e14-4d4b-4862-915e-eee1881c79d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:06.965108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad21e14-4d4b-4862-915e-eee1881c79d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:06.965414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad21e14-4d4b-4862-915e-eee1881c79d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:06.965414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad21e14-4d4b-4862-915e-eee1881c79d0 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:06.965414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 802/1600] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:06.967768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad21e14-4d4b-4862-915e-eee1881c79d0 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:06.984238 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad21e14-4d4b-4862-915e-eee1881c79d0 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:06.989172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebc1db-001a-479b-bfb0-a67f6e543bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:06.989790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebc1db-001a-479b-bfb0-a67f6e543bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:06.990084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebc1db-001a-479b-bfb0-a67f6e543bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:06.990340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ebc1db-001a-479b-bfb0-a67f6e543bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:06.990974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebc1db-001a-479b-bfb0-a67f6e543bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:06.992546 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ebc1db-001a-479b-bfb0-a67f6e543bb4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:07.008314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 803/1601] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:07.008314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebc1db-001a-479b-bfb0-a67f6e543bb4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:07.018477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ebc1db-001a-479b-bfb0-a67f6e543bb4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:07.023902 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60bf55-236d-4b33-bb28-84f1e2d2eca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:07.024575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60bf55-236d-4b33-bb28-84f1e2d2eca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:07.024850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60bf55-236d-4b33-bb28-84f1e2d2eca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:07.025135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60bf55-236d-4b33-bb28-84f1e2d2eca4 admin admin] PATH_INFO: `/v3/projects/90f04c5121d342c6ac5a1b135a8d7606/users/43b8fbf1dcc5464f872903a08950e0e1/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:07.042108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60bf55-236d-4b33-bb28-84f1e2d2eca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f04c5121d342c6ac5a1b135a8d7606, user_id=43b8fbf1dcc5464f872903a08950e0e1, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:07.045348 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60bf55-236d-4b33-bb28-84f1e2d2eca4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:07.235640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:07.407401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 804/1602] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:07 2022] PUT /identity/v3/projects/90f04c5121d342c6ac5a1b135a8d7606/users/43b8fbf1dcc5464f872903a08950e0e1/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:07.407663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60bf55-236d-4b33-bb28-84f1e2d2eca4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:07.416679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35689fe0ec754caf8be9a351f110cc3f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:07.417751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35689fe0ec754caf8be9a351f110cc3f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:07.506341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60bf55-236d-4b33-bb28-84f1e2d2eca4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:07.511246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f41922-ae99-498e-9ef9-8460d18e833e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:07.511916 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f41922-ae99-498e-9ef9-8460d18e833e admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:07.512195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f41922-ae99-498e-9ef9-8460d18e833e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:07.512467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f41922-ae99-498e-9ef9-8460d18e833e admin admin] PATH_INFO: `/v3/projects/7c3d7d79d1d3425ebf3d90d4c5006471/users/34df87739a2f4de0a8b290df6a524ec2/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:07.514132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35689fe0ec754caf8be9a351f110cc3f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:07.515516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35689fe0ec754caf8be9a351f110cc3f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:07.527244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35689fe0ec754caf8be9a351f110cc3f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:07.532231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f41922-ae99-498e-9ef9-8460d18e833e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c3d7d79d1d3425ebf3d90d4c5006471, user_id=34df87739a2f4de0a8b290df6a524ec2, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:07.535532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f41922-ae99-498e-9ef9-8460d18e833e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:07.549888 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 799/1603] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:06 2022] DELETE /identity/v3/users/35689fe0ec754caf8be9a351f110cc3f => generated 0 bytes in 1519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:07.550850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:07.552983 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:07.623826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 805/1604] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:07 2022] PUT /identity/v3/projects/7c3d7d79d1d3425ebf3d90d4c5006471/users/34df87739a2f4de0a8b290df6a524ec2/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:07.634557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9ec54-f76f-48d6-91c9-ddeb90d53b1a None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:07.634841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9ec54-f76f-48d6-91c9-ddeb90d53b1a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:07.635165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9ec54-f76f-48d6-91c9-ddeb90d53b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:07.679885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ccb9ec54-f76f-48d6-91c9-ddeb90d53b1a None None] MFA Rules not processed for user `34df87739a2f4de0a8b290df6a524ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:07.681158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ccb9ec54-f76f-48d6-91c9-ddeb90d53b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:07.687049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f0602-33c4-4bfe-8cb2-a8ba18a53921 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:07.744579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b927a7-35ab-49e4-80a6-242eb3d1818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:07.745194 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b927a7-35ab-49e4-80a6-242eb3d1818d admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:07.745466 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b927a7-35ab-49e4-80a6-242eb3d1818d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:07.745734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b927a7-35ab-49e4-80a6-242eb3d1818d admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:07.755017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 806/1605] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:07.758315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b927a7-35ab-49e4-80a6-242eb3d1818d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:07.759679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9ec54-f76f-48d6-91c9-ddeb90d53b1a None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:07.760020 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b927a7-35ab-49e4-80a6-242eb3d1818d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:07.761209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ccb9ec54-f76f-48d6-91c9-ddeb90d53b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:07.770698 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 800/1606] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:07.774829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b927a7-35ab-49e4-80a6-242eb3d1818d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:07.779263 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9ec54-f76f-48d6-91c9-ddeb90d53b1a None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:07.785404 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b927a7-35ab-49e4-80a6-242eb3d1818d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:07.785648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5bac8-206f-46a2-ab25-727ce7b63bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:07.786419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5bac8-206f-46a2-ab25-727ce7b63bbd admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:07.786782 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5bac8-206f-46a2-ab25-727ce7b63bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:07.787153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5bac8-206f-46a2-ab25-727ce7b63bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:07.787939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5bac8-206f-46a2-ab25-727ce7b63bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:07.789948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5bac8-206f-46a2-ab25-727ce7b63bbd admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:07.790482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3d4ea-beaa-423d-8fb2-1b3e12e7e03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:07.791224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3d4ea-beaa-423d-8fb2-1b3e12e7e03f admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:07.791606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3d4ea-beaa-423d-8fb2-1b3e12e7e03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:07.791977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3d4ea-beaa-423d-8fb2-1b3e12e7e03f admin admin] PATH_INFO: `/v3/projects/90f04c5121d342c6ac5a1b135a8d7606/users/43b8fbf1dcc5464f872903a08950e0e1/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:07.794098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3d4ea-beaa-423d-8fb2-1b3e12e7e03f admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f04c5121d342c6ac5a1b135a8d7606, user_id=43b8fbf1dcc5464f872903a08950e0e1, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:07.796828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3d4ea-beaa-423d-8fb2-1b3e12e7e03f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:07.810279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 801/1607] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:07 2022] PUT /identity/v3/projects/90f04c5121d342c6ac5a1b135a8d7606/users/43b8fbf1dcc5464f872903a08950e0e1/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:07.821843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef78d9e-b1d3-493d-a492-2fe7578096eb None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:07.822361 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef78d9e-b1d3-493d-a492-2fe7578096eb None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:07.822808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef78d9e-b1d3-493d-a492-2fe7578096eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:07.860157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-eef78d9e-b1d3-493d-a492-2fe7578096eb None None] MFA Rules not processed for user `43b8fbf1dcc5464f872903a08950e0e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:07.860157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-eef78d9e-b1d3-493d-a492-2fe7578096eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:07.924045 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 807/1608] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:07 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:33:07.925152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5bac8-206f-46a2-ab25-727ce7b63bbd admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:08.000983 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 802/1609] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:08.017576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef78d9e-b1d3-493d-a492-2fe7578096eb None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:08.022993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-eef78d9e-b1d3-493d-a492-2fe7578096eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:08.064433 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5bac8-206f-46a2-ab25-727ce7b63bbd admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:08.070844 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186014a1-74de-474d-a683-6d14e3467a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:08.071457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186014a1-74de-474d-a683-6d14e3467a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.071729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186014a1-74de-474d-a683-6d14e3467a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.072166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186014a1-74de-474d-a683-6d14e3467a4f admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.072807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186014a1-74de-474d-a683-6d14e3467a4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:08.075883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186014a1-74de-474d-a683-6d14e3467a4f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:08.105640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef78d9e-b1d3-493d-a492-2fe7578096eb None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:08.171637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 808/1610] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:07 2022] POST /identity/v3/users => generated 295 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:08.176208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186014a1-74de-474d-a683-6d14e3467a4f admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:08.189079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186014a1-74de-474d-a683-6d14e3467a4f admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:08.190329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08713d-c591-4c4b-81fc-03328bdc4120 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:08.191125 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08713d-c591-4c4b-81fc-03328bdc4120 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.195703 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08713d-c591-4c4b-81fc-03328bdc4120 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.196159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08713d-c591-4c4b-81fc-03328bdc4120 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.201900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2942bd-2b3e-4a3f-ab9c-cf8c77e69fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:08.202973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2942bd-2b3e-4a3f-ab9c-cf8c77e69fce admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.203473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2942bd-2b3e-4a3f-ab9c-cf8c77e69fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.203965 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2942bd-2b3e-4a3f-ab9c-cf8c77e69fce admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.205222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2942bd-2b3e-4a3f-ab9c-cf8c77e69fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:08.208432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2942bd-2b3e-4a3f-ab9c-cf8c77e69fce admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:08.233739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 809/1611] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:08.234885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2942bd-2b3e-4a3f-ab9c-cf8c77e69fce admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:08.237515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cf2942bd-2b3e-4a3f-ab9c-cf8c77e69fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:08.252128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2942bd-2b3e-4a3f-ab9c-cf8c77e69fce admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:08.258164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:08.259663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.259945 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.260218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] PATH_INFO: `/v3/projects/82753ee6eec04972a1888eecf140eb24` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.261374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=82753ee6eec04972a1888eecf140eb24)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:08.263731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:08.291347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08713d-c591-4c4b-81fc-03328bdc4120 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:08.292802 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08713d-c591-4c4b-81fc-03328bdc4120 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:08.302538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 803/1612] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:33:08.303300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08713d-c591-4c4b-81fc-03328bdc4120 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:08.315200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08713d-c591-4c4b-81fc-03328bdc4120 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:08.321116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f0542-af74-4e1e-bef6-4354869d4e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:08.321116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f0542-af74-4e1e-bef6-4354869d4e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.321116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f0542-af74-4e1e-bef6-4354869d4e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.321593 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f0542-af74-4e1e-bef6-4354869d4e13 admin admin] PATH_INFO: `/v3/projects/c3a6a5951fc0429b927cfbee2ed35c4c/users/6c6b9471db8646eca02ea686ed2d47bf/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.340458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f0542-af74-4e1e-bef6-4354869d4e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a6a5951fc0429b927cfbee2ed35c4c, user_id=6c6b9471db8646eca02ea686ed2d47bf, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:08.343647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f0542-af74-4e1e-bef6-4354869d4e13 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:08.451142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 804/1613] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:08 2022] PUT /identity/v3/projects/c3a6a5951fc0429b927cfbee2ed35c4c/users/6c6b9471db8646eca02ea686ed2d47bf/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:08.455133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f0542-af74-4e1e-bef6-4354869d4e13 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:08.461800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82753ee6eec04972a1888eecf140eb24', 'request_id': 'req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:08.470281 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82753ee6eec04972a1888eecf140eb24', 'request_id': 'req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:08.525186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f0542-af74-4e1e-bef6-4354869d4e13 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:08.582700 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3fce7-d933-4fab-ac27-32d1ace1001d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:08.583316 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3fce7-d933-4fab-ac27-32d1ace1001d admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.583640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3fce7-d933-4fab-ac27-32d1ace1001d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.583854 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3fce7-d933-4fab-ac27-32d1ace1001d admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.584513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3fce7-d933-4fab-ac27-32d1ace1001d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:08.586376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3fce7-d933-4fab-ac27-32d1ace1001d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:08.588920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:08.597921 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 805/1614] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:33:08.599751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3fce7-d933-4fab-ac27-32d1ace1001d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:08.621800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82753ee6eec04972a1888eecf140eb24', 'request_id': 'req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:08.635303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] The token cache is being invalidate because project 82753ee6eec04972a1888eecf140eb24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:08.635608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:08.636416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:08.638487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 810/1615] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:08 2022] DELETE /identity/v3/projects/82753ee6eec04972a1888eecf140eb24 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:08.642013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:08.642013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:08.655598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d2f6f-9edb-41ff-8e9b-d4370e7b2afd admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:08.660277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5141e4-989e-4f9e-bab7-eac1e6d5d485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:08.660808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3fce7-d933-4fab-ac27-32d1ace1001d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:08.660969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5141e4-989e-4f9e-bab7-eac1e6d5d485 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.661196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5141e4-989e-4f9e-bab7-eac1e6d5d485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.661464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5141e4-989e-4f9e-bab7-eac1e6d5d485 admin admin] PATH_INFO: `/v3/projects/c3a6a5951fc0429b927cfbee2ed35c4c/users/6c6b9471db8646eca02ea686ed2d47bf/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.663518 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5141e4-989e-4f9e-bab7-eac1e6d5d485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a6a5951fc0429b927cfbee2ed35c4c, user_id=6c6b9471db8646eca02ea686ed2d47bf, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:08.665028 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d806a-45d6-4c2e-8414-f21e877e7419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:08.665643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d806a-45d6-4c2e-8414-f21e877e7419 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.665912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d806a-45d6-4c2e-8414-f21e877e7419 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.666210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d806a-45d6-4c2e-8414-f21e877e7419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.666333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5141e4-989e-4f9e-bab7-eac1e6d5d485 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:08.667959 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-fe2d806a-45d6-4c2e-8414-f21e877e7419 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:08.679191 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 811/1616] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:08 2022] PUT /identity/v3/projects/c3a6a5951fc0429b927cfbee2ed35c4c/users/6c6b9471db8646eca02ea686ed2d47bf/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:08.681149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9aa6e-75cc-4a71-ac35-98caf11f3790 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.681431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9aa6e-75cc-4a71-ac35-98caf11f3790 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.681763 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9aa6e-75cc-4a71-ac35-98caf11f3790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.708486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-5af9aa6e-75cc-4a71-ac35-98caf11f3790 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:08.709913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5af9aa6e-75cc-4a71-ac35-98caf11f3790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:08.748860 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d806a-45d6-4c2e-8414-f21e877e7419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:08.750616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d806a-45d6-4c2e-8414-f21e877e7419 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:08.761731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 806/1617] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:08.763902 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff4b29-fa9e-4976-883c-227de39b19e1 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.764349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff4b29-fa9e-4976-883c-227de39b19e1 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.764762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff4b29-fa9e-4976-883c-227de39b19e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.807718 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 812/1618] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:08.811114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-65ff4b29-fa9e-4976-883c-227de39b19e1 None None] MFA Rules not processed for user `6c6b9471db8646eca02ea686ed2d47bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:08.812476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-65ff4b29-fa9e-4976-883c-227de39b19e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:08.820975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0bb1f-9b09-4434-81d4-d2aa83b41bc1 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.821267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0bb1f-9b09-4434-81d4-d2aa83b41bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.821540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0bb1f-9b09-4434-81d4-d2aa83b41bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.859323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-3ad0bb1f-9b09-4434-81d4-d2aa83b41bc1 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:08.860954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3ad0bb1f-9b09-4434-81d4-d2aa83b41bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:08.875071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 813/1619] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:08.876134 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0bb1f-9b09-4434-81d4-d2aa83b41bc1 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:08.886131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0bb1f-9b09-4434-81d4-d2aa83b41bc1 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:08.896974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94203b-fb8c-4626-88a7-7e036feadf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:08.898209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94203b-fb8c-4626-88a7-7e036feadf51 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.898690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94203b-fb8c-4626-88a7-7e036feadf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.899005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94203b-fb8c-4626-88a7-7e036feadf51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.904896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94203b-fb8c-4626-88a7-7e036feadf51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:33:08.904896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94203b-fb8c-4626-88a7-7e036feadf51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:08.904896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94203b-fb8c-4626-88a7-7e036feadf51 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:08.924822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 814/1620] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:33:08.925769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94203b-fb8c-4626-88a7-7e036feadf51 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:08.928850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 807/1621] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:08.948881 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff4b29-fa9e-4976-883c-227de39b19e1 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:08.955423 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-65ff4b29-fa9e-4976-883c-227de39b19e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:08.968157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94203b-fb8c-4626-88a7-7e036feadf51 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:08.974122 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee33402-f432-44ae-bcf5-e7d67810c0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:08.974856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee33402-f432-44ae-bcf5-e7d67810c0b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:08.975133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee33402-f432-44ae-bcf5-e7d67810c0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:08.975512 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee33402-f432-44ae-bcf5-e7d67810c0b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:08.976070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee33402-f432-44ae-bcf5-e7d67810c0b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:08.978163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee33402-f432-44ae-bcf5-e7d67810c0b6 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:09.047282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff4b29-fa9e-4976-883c-227de39b19e1 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:09.051750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f536c-e895-44a0-9346-888ca2aaa561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:09.052454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f536c-e895-44a0-9346-888ca2aaa561 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:09.052824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f536c-e895-44a0-9346-888ca2aaa561 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:09.053176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f536c-e895-44a0-9346-888ca2aaa561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:09.055578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-860f536c-e895-44a0-9346-888ca2aaa561 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:09.100348 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 815/1622] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:08 2022] POST /identity/v3/projects => generated 369 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:09.104789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee33402-f432-44ae-bcf5-e7d67810c0b6 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:09.119260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f536c-e895-44a0-9346-888ca2aaa561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:09.120743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f536c-e895-44a0-9346-888ca2aaa561 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:09.130263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 808/1623] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:09.172384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee33402-f432-44ae-bcf5-e7d67810c0b6 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:09.177074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b423e6e-062f-4baa-b00d-068b7e9d658d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:09.177677 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b423e6e-062f-4baa-b00d-068b7e9d658d admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:09.177946 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b423e6e-062f-4baa-b00d-068b7e9d658d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:09.178242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b423e6e-062f-4baa-b00d-068b7e9d658d admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:09.178997 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b423e6e-062f-4baa-b00d-068b7e9d658d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:09.181069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b423e6e-062f-4baa-b00d-068b7e9d658d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:09.265383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 816/1624] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:09 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:09.269056 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f536c-e895-44a0-9346-888ca2aaa561 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:09.282161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f536c-e895-44a0-9346-888ca2aaa561 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:09.287499 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263d1c3-c7f2-4497-8ee3-3dd69700029b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:09.288110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263d1c3-c7f2-4497-8ee3-3dd69700029b admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:09.288435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263d1c3-c7f2-4497-8ee3-3dd69700029b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:09.288727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2263d1c3-c7f2-4497-8ee3-3dd69700029b admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:09.289348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263d1c3-c7f2-4497-8ee3-3dd69700029b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:09.291015 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2263d1c3-c7f2-4497-8ee3-3dd69700029b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:09.301745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 809/1625] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:09.306022 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b423e6e-062f-4baa-b00d-068b7e9d658d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:09.329749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b423e6e-062f-4baa-b00d-068b7e9d658d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:09.347315 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4017213-7723-421a-b60c-b687b7c02f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:09.348065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4017213-7723-421a-b60c-b687b7c02f4c admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:09.351598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4017213-7723-421a-b60c-b687b7c02f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:09.351598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4017213-7723-421a-b60c-b687b7c02f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:09.351598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4017213-7723-421a-b60c-b687b7c02f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:09.352660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4017213-7723-421a-b60c-b687b7c02f4c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:09.367007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 817/1626] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:09.371059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263d1c3-c7f2-4497-8ee3-3dd69700029b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:09.383334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2263d1c3-c7f2-4497-8ee3-3dd69700029b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:09.388821 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187771d-4c7d-4d8e-92a1-19fab9690188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:09.389493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187771d-4c7d-4d8e-92a1-19fab9690188 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:09.389774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187771d-4c7d-4d8e-92a1-19fab9690188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:09.390068 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187771d-4c7d-4d8e-92a1-19fab9690188 admin admin] PATH_INFO: `/v3/projects/19e166a52dd5412499b10ecb92d1093a/users/5b1890fccda44fe6ba42ed3e1a5b3261/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:09.408496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187771d-4c7d-4d8e-92a1-19fab9690188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19e166a52dd5412499b10ecb92d1093a, user_id=5b1890fccda44fe6ba42ed3e1a5b3261, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:09.412144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187771d-4c7d-4d8e-92a1-19fab9690188 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:09.508382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 810/1627] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:09 2022] PUT /identity/v3/projects/19e166a52dd5412499b10ecb92d1093a/users/5b1890fccda44fe6ba42ed3e1a5b3261/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:09.519429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72454dd0-4056-49a8-8aae-918673adaaeb None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:09.519824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72454dd0-4056-49a8-8aae-918673adaaeb None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:09.520165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72454dd0-4056-49a8-8aae-918673adaaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:09.549936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-72454dd0-4056-49a8-8aae-918673adaaeb None None] MFA Rules not processed for user `5b1890fccda44fe6ba42ed3e1a5b3261`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:09.551235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-72454dd0-4056-49a8-8aae-918673adaaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:09.618662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 818/1628] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:09.623497 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187771d-4c7d-4d8e-92a1-19fab9690188 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:09.656164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72454dd0-4056-49a8-8aae-918673adaaeb None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:09.661870 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:09.663017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-72454dd0-4056-49a8-8aae-918673adaaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:09.719030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187771d-4c7d-4d8e-92a1-19fab9690188 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:09.735999 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13cd66-8434-43be-a269-9404d28e5310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:09.736389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13cd66-8434-43be-a269-9404d28e5310 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:09.736726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13cd66-8434-43be-a269-9404d28e5310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:09.737240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13cd66-8434-43be-a269-9404d28e5310 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:09.737928 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13cd66-8434-43be-a269-9404d28e5310 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:09.739918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13cd66-8434-43be-a269-9404d28e5310 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:09.745505 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72454dd0-4056-49a8-8aae-918673adaaeb None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:09.750603 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:09.753201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:09.753201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:09.753201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] PATH_INFO: `/v3/users/2e9d94307a01406db599522516094e56` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:09.754194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e9d94307a01406db599522516094e56)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:09.756184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:09.902775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 811/1629] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:09 2022] POST /identity/v3/projects => generated 369 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:09.902775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13cd66-8434-43be-a269-9404d28e5310 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:09.904478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ec13cd66-8434-43be-a269-9404d28e5310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:10.000689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13cd66-8434-43be-a269-9404d28e5310 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:10.011470 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:10.012147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.017228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.017228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] PATH_INFO: `/v3/users/ab0d96ea9de84797a7d3cb233799f850` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.019850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0d96ea9de84797a7d3cb233799f850)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:10.022352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:10.053779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e9d94307a01406db599522516094e56'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:10.055189 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e9d94307a01406db599522516094e56'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:10.064030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e9d94307a01406db599522516094e56'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:10.148036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e9d94307a01406db599522516094e56'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:10.148673 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e9d94307a01406db599522516094e56'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:10.168505 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 819/1630] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:09 2022] DELETE /identity/v3/users/2e9d94307a01406db599522516094e56 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:10.169529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:10.171286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:10.246890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0d96ea9de84797a7d3cb233799f850'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:10.247579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0d96ea9de84797a7d3cb233799f850'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:10.261017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec395e15-eba4-49c6-8315-cf89c47096bf admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:10.267565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a991ab-3389-486e-86b9-6502697de21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:10.268576 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a991ab-3389-486e-86b9-6502697de21f admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.269660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a991ab-3389-486e-86b9-6502697de21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.270148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a991ab-3389-486e-86b9-6502697de21f admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.271307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a991ab-3389-486e-86b9-6502697de21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:10.274873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a991ab-3389-486e-86b9-6502697de21f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:10.372263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0d96ea9de84797a7d3cb233799f850'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:10.375380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0d96ea9de84797a7d3cb233799f850'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:10.380030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 820/1631] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:10 2022] POST /identity/v3/users => generated 291 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:10.382294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3659f1-2661-4bc5-a525-fac1de889d7c None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.382599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3659f1-2661-4bc5-a525-fac1de889d7c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.382868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3659f1-2661-4bc5-a525-fac1de889d7c None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.389621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 821/1632] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:33:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:33:10.392045 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9eba32-54fa-4965-b115-11bc0c0f8238 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.392289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9eba32-54fa-4965-b115-11bc0c0f8238 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.392615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9eba32-54fa-4965-b115-11bc0c0f8238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.392912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0d96ea9de84797a7d3cb233799f850'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:10.429230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 812/1633] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:09 2022] DELETE /identity/v3/users/ab0d96ea9de84797a7d3cb233799f850 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:10.430348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:10.431910 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:10.434339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-8f9eba32-54fa-4965-b115-11bc0c0f8238 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:10.436344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8f9eba32-54fa-4965-b115-11bc0c0f8238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:10.516567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 822/1634] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:10.516567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54fe3c-43bf-4121-b1d2-75a99e1f985f None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.516567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54fe3c-43bf-4121-b1d2-75a99e1f985f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.516567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54fe3c-43bf-4121-b1d2-75a99e1f985f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.537489 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e8e2f-455c-4440-b433-2f14b1f2e185 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:10.547311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-eb54fe3c-43bf-4121-b1d2-75a99e1f985f None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:10.548530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-eb54fe3c-43bf-4121-b1d2-75a99e1f985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:10.555777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69440550-4bb7-4f29-9132-a2ef22191899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:10.556857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69440550-4bb7-4f29-9132-a2ef22191899 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.557291 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69440550-4bb7-4f29-9132-a2ef22191899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.557705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69440550-4bb7-4f29-9132-a2ef22191899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.558530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69440550-4bb7-4f29-9132-a2ef22191899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:10.561149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69440550-4bb7-4f29-9132-a2ef22191899 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:10.592478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 813/1635] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:10.592478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44bf51-2570-4069-ab1b-ff37630e23df None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.592478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44bf51-2570-4069-ab1b-ff37630e23df None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.592478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44bf51-2570-4069-ab1b-ff37630e23df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.612408 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-2c44bf51-2570-4069-ab1b-ff37630e23df None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:10.614133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2c44bf51-2570-4069-ab1b-ff37630e23df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:10.623403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 814/1636] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:10.624269 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44bf51-2570-4069-ab1b-ff37630e23df None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:10.625625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2c44bf51-2570-4069-ab1b-ff37630e23df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:10.682292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 823/1637] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:33:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:10.687107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54fe3c-43bf-4121-b1d2-75a99e1f985f None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:10.701319 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44bf51-2570-4069-ab1b-ff37630e23df None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:10.701518 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54fe3c-43bf-4121-b1d2-75a99e1f985f None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:10.707377 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f2443-f87d-4bfc-829e-0e8295649619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:10.707613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba48ede-9a5e-424a-a351-bc7cf3dc4500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:10.708205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f2443-f87d-4bfc-829e-0e8295649619 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.708316 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba48ede-9a5e-424a-a351-bc7cf3dc4500 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.708567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba48ede-9a5e-424a-a351-bc7cf3dc4500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.708684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f2443-f87d-4bfc-829e-0e8295649619 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.708895 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba48ede-9a5e-424a-a351-bc7cf3dc4500 admin admin] PATH_INFO: `/v3/projects/3553bdbadbfa4cbea478779087052022/users/1ca46003975d4aed94a5ef2a1e9020a4/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.709086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f2443-f87d-4bfc-829e-0e8295649619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.722253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f2443-f87d-4bfc-829e-0e8295649619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:10.724338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f2443-f87d-4bfc-829e-0e8295649619 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:10.726943 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba48ede-9a5e-424a-a351-bc7cf3dc4500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3553bdbadbfa4cbea478779087052022, user_id=1ca46003975d4aed94a5ef2a1e9020a4, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:10.729118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba48ede-9a5e-424a-a351-bc7cf3dc4500 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:10.735769 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 815/1638] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:10.736849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f2443-f87d-4bfc-829e-0e8295649619 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:10.738387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7f4f2443-f87d-4bfc-829e-0e8295649619 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:10.753295 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f2443-f87d-4bfc-829e-0e8295649619 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:10.757365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c115a8-9ee0-4b69-8503-653f140103f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:10.758706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c115a8-9ee0-4b69-8503-653f140103f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.758997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c115a8-9ee0-4b69-8503-653f140103f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.759379 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c115a8-9ee0-4b69-8503-653f140103f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.770261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c115a8-9ee0-4b69-8503-653f140103f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:10.771701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c115a8-9ee0-4b69-8503-653f140103f2 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:10.782069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 816/1639] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:10.783087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c115a8-9ee0-4b69-8503-653f140103f2 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:10.784980 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e8c115a8-9ee0-4b69-8503-653f140103f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:10.801752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c115a8-9ee0-4b69-8503-653f140103f2 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:10.808866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9671c16-1575-4fd1-b225-15f2f83e2668 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:10.813394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671c16-1575-4fd1-b225-15f2f83e2668 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.813394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671c16-1575-4fd1-b225-15f2f83e2668 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.813394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9671c16-1575-4fd1-b225-15f2f83e2668 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.834117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9671c16-1575-4fd1-b225-15f2f83e2668 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:10.835964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9671c16-1575-4fd1-b225-15f2f83e2668 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:10.862695 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 824/1640] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:10 2022] PUT /identity/v3/projects/3553bdbadbfa4cbea478779087052022/users/1ca46003975d4aed94a5ef2a1e9020a4/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:10.869364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 817/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:10.869563 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba48ede-9a5e-424a-a351-bc7cf3dc4500 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:10.980125 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba48ede-9a5e-424a-a351-bc7cf3dc4500 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:10.986930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b777d-3c18-4d79-8287-1f06a3731d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:10.987618 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b777d-3c18-4d79-8287-1f06a3731d5d admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:10.987904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b777d-3c18-4d79-8287-1f06a3731d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:10.988178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b777d-3c18-4d79-8287-1f06a3731d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:10.988949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b777d-3c18-4d79-8287-1f06a3731d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:10.990706 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b777d-3c18-4d79-8287-1f06a3731d5d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:11.017739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 825/1642] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:11.021954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9671c16-1575-4fd1-b225-15f2f83e2668 service nova] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:11.033893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9671c16-1575-4fd1-b225-15f2f83e2668 service nova] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:11.041088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a39642-8b44-4177-ab8d-b32a4b6a1217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:11.041901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a39642-8b44-4177-ab8d-b32a4b6a1217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:11.042322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a39642-8b44-4177-ab8d-b32a4b6a1217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:11.042724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a39642-8b44-4177-ab8d-b32a4b6a1217 admin admin] PATH_INFO: `/v3/projects/3553bdbadbfa4cbea478779087052022/users/1ca46003975d4aed94a5ef2a1e9020a4/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:11.045445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a39642-8b44-4177-ab8d-b32a4b6a1217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3553bdbadbfa4cbea478779087052022, user_id=1ca46003975d4aed94a5ef2a1e9020a4, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:11.048283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a39642-8b44-4177-ab8d-b32a4b6a1217 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:11.061174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 818/1643] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:11 2022] PUT /identity/v3/projects/3553bdbadbfa4cbea478779087052022/users/1ca46003975d4aed94a5ef2a1e9020a4/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:11.072915 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b59f2-11f5-450d-a735-7008c4589d6c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:11.073419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b59f2-11f5-450d-a735-7008c4589d6c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:11.073491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b59f2-11f5-450d-a735-7008c4589d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:11.119865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:11.126778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-207b59f2-11f5-450d-a735-7008c4589d6c None None] MFA Rules not processed for user `1ca46003975d4aed94a5ef2a1e9020a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:11.128152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-207b59f2-11f5-450d-a735-7008c4589d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:11.226232 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 826/1644] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:11.238911 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a39642-8b44-4177-ab8d-b32a4b6a1217 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:11.240857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-95a39642-8b44-4177-ab8d-b32a4b6a1217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:11.302450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a39642-8b44-4177-ab8d-b32a4b6a1217 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:11.306237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72841794-553c-4bb5-a8fe-063c22101f41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:11.306863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72841794-553c-4bb5-a8fe-063c22101f41 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:11.307140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72841794-553c-4bb5-a8fe-063c22101f41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:11.307412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72841794-553c-4bb5-a8fe-063c22101f41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:11.316725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72841794-553c-4bb5-a8fe-063c22101f41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:11.318189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72841794-553c-4bb5-a8fe-063c22101f41 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:11.327627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 819/1645] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:11.358621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b59f2-11f5-450d-a735-7008c4589d6c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:11.359963 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-207b59f2-11f5-450d-a735-7008c4589d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:11.388751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72841794-553c-4bb5-a8fe-063c22101f41 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:11.393594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-72841794-553c-4bb5-a8fe-063c22101f41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:11.491080 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:11.498245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b59f2-11f5-450d-a735-7008c4589d6c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:11.504704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:11.505434 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:11.505808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:11.505975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] PATH_INFO: `/v3/projects/bb0a1301ccd1479a864b58d704b8421b` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:11.507413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb0a1301ccd1479a864b58d704b8421b)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:11.509565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:11.514020 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72841794-553c-4bb5-a8fe-063c22101f41 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:11.519085 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:11.520263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:11.520263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:11.520769 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] PATH_INFO: `/v3/projects/fd02c7532ef24674979cb3cbd2194bff` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:11.521928 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd02c7532ef24674979cb3cbd2194bff)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:11.524197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:11.689169 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd02c7532ef24674979cb3cbd2194bff', 'request_id': 'req-dc4f8008-8e31-43fd-80db-9342df2bee87'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:11.698478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd02c7532ef24674979cb3cbd2194bff', 'request_id': 'req-dc4f8008-8e31-43fd-80db-9342df2bee87'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:11.699292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb0a1301ccd1479a864b58d704b8421b', 'request_id': 'req-ecd1053b-ee91-46bc-8f49-3abe4726b286'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:11.712634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb0a1301ccd1479a864b58d704b8421b', 'request_id': 'req-ecd1053b-ee91-46bc-8f49-3abe4726b286'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:11.845035 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd02c7532ef24674979cb3cbd2194bff', 'request_id': 'req-dc4f8008-8e31-43fd-80db-9342df2bee87'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:11.856760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] The token cache is being invalidate because project fd02c7532ef24674979cb3cbd2194bff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:11.857121 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:11.857849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:11.862591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 820/1646] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:11 2022] DELETE /identity/v3/projects/fd02c7532ef24674979cb3cbd2194bff => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:11.862591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:11.862591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:11.927385 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f8008-8e31-43fd-80db-9342df2bee87 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:11.931646 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be7c26-1e02-4570-a081-969af48c0f03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:11.932281 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be7c26-1e02-4570-a081-969af48c0f03 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:11.932562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be7c26-1e02-4570-a081-969af48c0f03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:11.932837 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be7c26-1e02-4570-a081-969af48c0f03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:11.935056 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-43be7c26-1e02-4570-a081-969af48c0f03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:11.948612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb0a1301ccd1479a864b58d704b8421b', 'request_id': 'req-ecd1053b-ee91-46bc-8f49-3abe4726b286'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:11.971753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] The token cache is being invalidate because project bb0a1301ccd1479a864b58d704b8421b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:11.972856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:11.974231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:11.979864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 827/1647] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:11 2022] DELETE /identity/v3/projects/bb0a1301ccd1479a864b58d704b8421b => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:11.981234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:11.984247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:12.027699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be7c26-1e02-4570-a081-969af48c0f03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:12.031925 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be7c26-1e02-4570-a081-969af48c0f03 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:12.048987 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 821/1648] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:12.049739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be7c26-1e02-4570-a081-969af48c0f03 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:12.051147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-43be7c26-1e02-4570-a081-969af48c0f03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:12.115402 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1053b-ee91-46bc-8f49-3abe4726b286 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:12.188575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:12.189112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:12.189411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:12.189701 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] PATH_INFO: `/v3/users/dacc58cbc3614a1c9e1caed5af6efd5e` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:12.190957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dacc58cbc3614a1c9e1caed5af6efd5e)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:12.192838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be7c26-1e02-4570-a081-969af48c0f03 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:12.193159 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:12.200882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:12.200882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:12.200882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:12.200882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] PATH_INFO: `/v3/users/43b8fbf1dcc5464f872903a08950e0e1` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:12.201366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] RBAC: Authorizing `identity:delete_user(user_id=43b8fbf1dcc5464f872903a08950e0e1)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:12.203958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:12.391667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dacc58cbc3614a1c9e1caed5af6efd5e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:12.392299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dacc58cbc3614a1c9e1caed5af6efd5e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:12.400018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dacc58cbc3614a1c9e1caed5af6efd5e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:12.422159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43b8fbf1dcc5464f872903a08950e0e1'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:12.423123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43b8fbf1dcc5464f872903a08950e0e1'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:12.541620 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dacc58cbc3614a1c9e1caed5af6efd5e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:12.542294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dacc58cbc3614a1c9e1caed5af6efd5e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:12.569516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 828/1649] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:11 2022] DELETE /identity/v3/users/dacc58cbc3614a1c9e1caed5af6efd5e => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:12.696104 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43b8fbf1dcc5464f872903a08950e0e1'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:12.697243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43b8fbf1dcc5464f872903a08950e0e1'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:12.706480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43b8fbf1dcc5464f872903a08950e0e1'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:12.725075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:12.725626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 822/1650] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:12 2022] DELETE /identity/v3/users/43b8fbf1dcc5464f872903a08950e0e1 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:13.146303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:13.146303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:13.227936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1090970-ef1c-474a-921d-4ac0dabd5ab0 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:13.299549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:13.300231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:13.300506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:13.300775 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] PATH_INFO: `/v3/projects/32bb9868f33647678ecf8eca89d7ba42` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:13.302406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32bb9868f33647678ecf8eca89d7ba42)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:13.304321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:13.355630 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:13.358622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:13.377183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8838f-18f9-4ec5-b1cf-34a8e196214d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:13.379495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:13.389954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:13.390638 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:13.390921 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:13.391194 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] PATH_INFO: `/v3/projects/90f04c5121d342c6ac5a1b135a8d7606` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:13.392355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f04c5121d342c6ac5a1b135a8d7606)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:13.394438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:13.461724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32bb9868f33647678ecf8eca89d7ba42', 'request_id': 'req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:13.468411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32bb9868f33647678ecf8eca89d7ba42', 'request_id': 'req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:13.537675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f04c5121d342c6ac5a1b135a8d7606', 'request_id': 'req-17f1729b-b52b-465c-9460-ecc6ffd77983'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:13.544895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f04c5121d342c6ac5a1b135a8d7606', 'request_id': 'req-17f1729b-b52b-465c-9460-ecc6ffd77983'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:13.576507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32bb9868f33647678ecf8eca89d7ba42', 'request_id': 'req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:13.588669 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] The token cache is being invalidate because project 32bb9868f33647678ecf8eca89d7ba42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:13.588959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:13.589615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d9e2381d-d9f9-4ab9-a9d8-672675e1c968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:13.591829 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 829/1651] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:13 2022] DELETE /identity/v3/projects/32bb9868f33647678ecf8eca89d7ba42 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:13.596486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23ed63-3649-4332-b089-45f0642c5335 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:13.596769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23ed63-3649-4332-b089-45f0642c5335 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:13.597067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23ed63-3649-4332-b089-45f0642c5335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:13.617209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ec23ed63-3649-4332-b089-45f0642c5335 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:13.618466 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ec23ed63-3649-4332-b089-45f0642c5335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:13.675128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 830/1652] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:13.682773 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b73ae-72d5-4c3e-9bbd-5f3585476bc3 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:13.683063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b73ae-72d5-4c3e-9bbd-5f3585476bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:13.683339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b73ae-72d5-4c3e-9bbd-5f3585476bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:13.701801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-166b73ae-72d5-4c3e-9bbd-5f3585476bc3 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:13.703034 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-166b73ae-72d5-4c3e-9bbd-5f3585476bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:13.711129 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 831/1653] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:13.721014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b73ae-72d5-4c3e-9bbd-5f3585476bc3 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:13.724911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b73ae-72d5-4c3e-9bbd-5f3585476bc3 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:13.729596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de172113-39b1-44b1-9b9c-67626a6f5509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:13.730232 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de172113-39b1-44b1-9b9c-67626a6f5509 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:13.733437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de172113-39b1-44b1-9b9c-67626a6f5509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:13.733437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de172113-39b1-44b1-9b9c-67626a6f5509 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:13.733437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de172113-39b1-44b1-9b9c-67626a6f5509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:33:13.733437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de172113-39b1-44b1-9b9c-67626a6f5509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:13.733912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de172113-39b1-44b1-9b9c-67626a6f5509 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:13.745220 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 832/1654] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:33:13.749266 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de172113-39b1-44b1-9b9c-67626a6f5509 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:13.753400 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f04c5121d342c6ac5a1b135a8d7606', 'request_id': 'req-17f1729b-b52b-465c-9460-ecc6ffd77983'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:13.758807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de172113-39b1-44b1-9b9c-67626a6f5509 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:13.763389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bac0e8-4221-4f97-ae90-a9f394bab5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:13.763953 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bac0e8-4221-4f97-ae90-a9f394bab5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:13.764204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] The token cache is being invalidate because project 90f04c5121d342c6ac5a1b135a8d7606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:13.764300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bac0e8-4221-4f97-ae90-a9f394bab5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:13.764491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bac0e8-4221-4f97-ae90-a9f394bab5c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:13.764578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:13.765101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bac0e8-4221-4f97-ae90-a9f394bab5c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:13.765190 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-17f1729b-b52b-465c-9460-ecc6ffd77983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:13.766625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 823/1655] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:13 2022] DELETE /identity/v3/projects/90f04c5121d342c6ac5a1b135a8d7606 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:13.766992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bac0e8-4221-4f97-ae90-a9f394bab5c6 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:13.771313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fdc3b-22bc-4ad4-aa42-c9b9cb360aa1 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:13.771684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fdc3b-22bc-4ad4-aa42-c9b9cb360aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:13.772018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fdc3b-22bc-4ad4-aa42-c9b9cb360aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:13.792871 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-d84fdc3b-22bc-4ad4-aa42-c9b9cb360aa1 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:13.794135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d84fdc3b-22bc-4ad4-aa42-c9b9cb360aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:13.802661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 824/1656] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:13.810293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e22c7-1db7-4c7a-bbe7-c90ed96a3de1 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:13.810491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e22c7-1db7-4c7a-bbe7-c90ed96a3de1 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:13.810769 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e22c7-1db7-4c7a-bbe7-c90ed96a3de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:13.830560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-ed3e22c7-1db7-4c7a-bbe7-c90ed96a3de1 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:13.831760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ed3e22c7-1db7-4c7a-bbe7-c90ed96a3de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:13.840154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 825/1657] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:13.844799 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e22c7-1db7-4c7a-bbe7-c90ed96a3de1 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:13.853790 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e22c7-1db7-4c7a-bbe7-c90ed96a3de1 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:13.858984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab975f14-336d-43af-b518-f75ef27738f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:13.859572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab975f14-336d-43af-b518-f75ef27738f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:13.859847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab975f14-336d-43af-b518-f75ef27738f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:13.862617 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab975f14-336d-43af-b518-f75ef27738f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:13.863201 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab975f14-336d-43af-b518-f75ef27738f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:33:13.863637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab975f14-336d-43af-b518-f75ef27738f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:13.865463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab975f14-336d-43af-b518-f75ef27738f0 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:13.875872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 833/1658] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:13 2022] POST /identity/v3/projects => generated 371 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:33:13.879249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 826/1659] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:33:13.880172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bac0e8-4221-4f97-ae90-a9f394bab5c6 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:13.883286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab975f14-336d-43af-b518-f75ef27738f0 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:13.884520 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a3bac0e8-4221-4f97-ae90-a9f394bab5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:13.894009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab975f14-336d-43af-b518-f75ef27738f0 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:13.958492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bac0e8-4221-4f97-ae90-a9f394bab5c6 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:13.959838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d61f5-e010-4a9c-a2be-1c3df1e4a152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:13.960642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d61f5-e010-4a9c-a2be-1c3df1e4a152 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:13.960977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d61f5-e010-4a9c-a2be-1c3df1e4a152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:13.961195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d61f5-e010-4a9c-a2be-1c3df1e4a152 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:13.961829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d61f5-e010-4a9c-a2be-1c3df1e4a152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:13.963881 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d61f5-e010-4a9c-a2be-1c3df1e4a152 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:13.964413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0347e-ab66-4f78-a5a9-63aff2c7fc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:13.965114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0347e-ab66-4f78-a5a9-63aff2c7fc5e admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:13.965455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0347e-ab66-4f78-a5a9-63aff2c7fc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:13.965790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0347e-ab66-4f78-a5a9-63aff2c7fc5e admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:13.967094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0347e-ab66-4f78-a5a9-63aff2c7fc5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:13.969142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0347e-ab66-4f78-a5a9-63aff2c7fc5e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:14.073115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 834/1660] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:13 2022] POST /identity/v3/users => generated 286 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:14.080267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0347e-ab66-4f78-a5a9-63aff2c7fc5e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:14.090345 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0347e-ab66-4f78-a5a9-63aff2c7fc5e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:14.095574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409471b-fb6d-4e42-ba8f-eb3804b6034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:14.096173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409471b-fb6d-4e42-ba8f-eb3804b6034c admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:14.096466 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409471b-fb6d-4e42-ba8f-eb3804b6034c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:14.096723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409471b-fb6d-4e42-ba8f-eb3804b6034c admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:14.097371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409471b-fb6d-4e42-ba8f-eb3804b6034c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:14.098993 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409471b-fb6d-4e42-ba8f-eb3804b6034c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:14.109800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 835/1661] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:33:14.117578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 827/1662] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:13 2022] POST /identity/v3/projects => generated 373 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:33:14.119723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409471b-fb6d-4e42-ba8f-eb3804b6034c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:14.121554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d61f5-e010-4a9c-a2be-1c3df1e4a152 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:14.130341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409471b-fb6d-4e42-ba8f-eb3804b6034c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:14.182260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d61f5-e010-4a9c-a2be-1c3df1e4a152 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:14.185162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909535fe-7347-474a-be95-cdf7b0c97bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:14.185791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909535fe-7347-474a-be95-cdf7b0c97bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:14.186101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909535fe-7347-474a-be95-cdf7b0c97bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:14.186377 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909535fe-7347-474a-be95-cdf7b0c97bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:14.187020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909535fe-7347-474a-be95-cdf7b0c97bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:14.187129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724640b-6974-4470-9b69-79a7e1ba1e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:14.187697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724640b-6974-4470-9b69-79a7e1ba1e7b admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:14.187971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724640b-6974-4470-9b69-79a7e1ba1e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:14.188241 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f724640b-6974-4470-9b69-79a7e1ba1e7b admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:14.188592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909535fe-7347-474a-be95-cdf7b0c97bb3 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:14.188859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724640b-6974-4470-9b69-79a7e1ba1e7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:14.190914 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f724640b-6974-4470-9b69-79a7e1ba1e7b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:14.203289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 836/1663] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:14.209629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909535fe-7347-474a-be95-cdf7b0c97bb3 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:14.219897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909535fe-7347-474a-be95-cdf7b0c97bb3 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:14.225851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef10da-e66e-4b3e-ad4d-21120953cc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:14.229642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef10da-e66e-4b3e-ad4d-21120953cc7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:14.229642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef10da-e66e-4b3e-ad4d-21120953cc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:14.229642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef10da-e66e-4b3e-ad4d-21120953cc7e admin admin] PATH_INFO: `/v3/projects/25e6066db9924f9a9e39a4c1f0356459/users/22919b0a4b1e4e09b9796a93539547b6/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:14.242862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef10da-e66e-4b3e-ad4d-21120953cc7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e6066db9924f9a9e39a4c1f0356459, user_id=22919b0a4b1e4e09b9796a93539547b6, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:14.245467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef10da-e66e-4b3e-ad4d-21120953cc7e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:14.330203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 828/1664] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:14 2022] POST /identity/v3/users => generated 287 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:33:14.331156 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724640b-6974-4470-9b69-79a7e1ba1e7b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:14.343770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f724640b-6974-4470-9b69-79a7e1ba1e7b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:14.350970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a536469-cbde-4adc-bc2d-2bdbec9a1989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:14.353665 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536469-cbde-4adc-bc2d-2bdbec9a1989 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:14.354150 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536469-cbde-4adc-bc2d-2bdbec9a1989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:14.354432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536469-cbde-4adc-bc2d-2bdbec9a1989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:14.355106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a536469-cbde-4adc-bc2d-2bdbec9a1989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:14.357057 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a536469-cbde-4adc-bc2d-2bdbec9a1989 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:14.365072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 837/1665] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:14 2022] PUT /identity/v3/projects/25e6066db9924f9a9e39a4c1f0356459/users/22919b0a4b1e4e09b9796a93539547b6/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:14.369597 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 829/1666] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:33:14.373538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd7270-5490-4ba4-a7ef-b032ef74de8d None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:14.373826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd7270-5490-4ba4-a7ef-b032ef74de8d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:14.374126 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd7270-5490-4ba4-a7ef-b032ef74de8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:14.374511 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a536469-cbde-4adc-bc2d-2bdbec9a1989 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:14.403316 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-f0dd7270-5490-4ba4-a7ef-b032ef74de8d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:14.404628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f0dd7270-5490-4ba4-a7ef-b032ef74de8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:14.407776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a536469-cbde-4adc-bc2d-2bdbec9a1989 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:14.445501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:14.494087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee115d2-739e-4568-aa50-f2fe69995f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:14.494692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee115d2-739e-4568-aa50-f2fe69995f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:14.495033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee115d2-739e-4568-aa50-f2fe69995f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:14.495309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee115d2-739e-4568-aa50-f2fe69995f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:14.495945 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee115d2-739e-4568-aa50-f2fe69995f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:14.498362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee115d2-739e-4568-aa50-f2fe69995f32 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:14.509640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 830/1667] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:33:14.510394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 838/1668] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:14.519968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd7270-5490-4ba4-a7ef-b032ef74de8d None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:14.520493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee115d2-739e-4568-aa50-f2fe69995f32 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:14.521642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f0dd7270-5490-4ba4-a7ef-b032ef74de8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:14.537535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee115d2-739e-4568-aa50-f2fe69995f32 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:14.543468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3947d5d-c6f8-44db-a06c-c3f88850e65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:14.544414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3947d5d-c6f8-44db-a06c-c3f88850e65d admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:14.544852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3947d5d-c6f8-44db-a06c-c3f88850e65d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:14.545278 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3947d5d-c6f8-44db-a06c-c3f88850e65d admin admin] PATH_INFO: `/v3/projects/ef4a7a64959d4cc99f25337a45d634be/users/cfa9afff01b3432790a6b63f72a45c96/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:14.569780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3947d5d-c6f8-44db-a06c-c3f88850e65d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4a7a64959d4cc99f25337a45d634be, user_id=cfa9afff01b3432790a6b63f72a45c96, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:14.572677 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3947d5d-c6f8-44db-a06c-c3f88850e65d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:14.605998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd7270-5490-4ba4-a7ef-b032ef74de8d None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:14.611741 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105d544-559c-4c89-8bab-d6ac331ff5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:14.612397 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105d544-559c-4c89-8bab-d6ac331ff5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:14.612675 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105d544-559c-4c89-8bab-d6ac331ff5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:14.612950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105d544-559c-4c89-8bab-d6ac331ff5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:14.624120 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105d544-559c-4c89-8bab-d6ac331ff5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:14.625608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105d544-559c-4c89-8bab-d6ac331ff5f6 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:14.636206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 839/1669] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:14.696083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 831/1670] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:14 2022] PUT /identity/v3/projects/ef4a7a64959d4cc99f25337a45d634be/users/cfa9afff01b3432790a6b63f72a45c96/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:14.701904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabff5d-95e9-4258-8da0-e509c75512fd None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:14.701904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabff5d-95e9-4258-8da0-e509c75512fd None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:14.705465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabff5d-95e9-4258-8da0-e509c75512fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:14.722613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2aabff5d-95e9-4258-8da0-e509c75512fd None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:14.724083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2aabff5d-95e9-4258-8da0-e509c75512fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:14.788608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 840/1671] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:14.804521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3947d5d-c6f8-44db-a06c-c3f88850e65d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:14.867660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3947d5d-c6f8-44db-a06c-c3f88850e65d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:14.873439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256aa51b-d6ef-47f4-a759-3561f6f545c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:14.874192 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256aa51b-d6ef-47f4-a759-3561f6f545c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:14.874495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256aa51b-d6ef-47f4-a759-3561f6f545c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:14.874764 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256aa51b-d6ef-47f4-a759-3561f6f545c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:14.884552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256aa51b-d6ef-47f4-a759-3561f6f545c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:14.886027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256aa51b-d6ef-47f4-a759-3561f6f545c5 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:14.895811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 832/1672] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:15.536685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:15.812358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabff5d-95e9-4258-8da0-e509c75512fd None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:15.814123 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2aabff5d-95e9-4258-8da0-e509c75512fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:15.887399 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabff5d-95e9-4258-8da0-e509c75512fd None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:15.894358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817a234-e675-4592-9150-c7b6b1826c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:15.895677 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817a234-e675-4592-9150-c7b6b1826c8d service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:15.896083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817a234-e675-4592-9150-c7b6b1826c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:15.896458 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817a234-e675-4592-9150-c7b6b1826c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:15.901552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d817a234-e675-4592-9150-c7b6b1826c8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:15.980549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817a234-e675-4592-9150-c7b6b1826c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:15.982105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817a234-e675-4592-9150-c7b6b1826c8d service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:15.992276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 841/1673] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:16.388579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:16.640844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dacb88-f574-4f60-937e-f8518e0969d0 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:16.641634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dacb88-f574-4f60-937e-f8518e0969d0 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:16.642260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dacb88-f574-4f60-937e-f8518e0969d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:16.672138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-d2dacb88-f574-4f60-937e-f8518e0969d0 None None] MFA Rules not processed for user `22919b0a4b1e4e09b9796a93539547b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:16.673362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d2dacb88-f574-4f60-937e-f8518e0969d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:16.743515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 833/1674] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:16.755659 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219d5d9-f7bf-4631-af9d-dc23f75d779e None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:16.756074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219d5d9-f7bf-4631-af9d-dc23f75d779e None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:16.756470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219d5d9-f7bf-4631-af9d-dc23f75d779e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:16.787576 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-4219d5d9-f7bf-4631-af9d-dc23f75d779e None None] MFA Rules not processed for user `cfa9afff01b3432790a6b63f72a45c96`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:16.788877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4219d5d9-f7bf-4631-af9d-dc23f75d779e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:16.885183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dacb88-f574-4f60-937e-f8518e0969d0 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:16.896256 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 842/1675] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:16.896743 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219d5d9-f7bf-4631-af9d-dc23f75d779e None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:16.900106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dacb88-f574-4f60-937e-f8518e0969d0 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:16.907580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b6044-ba06-4f6d-9744-bed96e3a7cd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:16.909012 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b6044-ba06-4f6d-9744-bed96e3a7cd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:16.909652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b6044-ba06-4f6d-9744-bed96e3a7cd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:16.911798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b6044-ba06-4f6d-9744-bed96e3a7cd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:16.913417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219d5d9-f7bf-4631-af9d-dc23f75d779e None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:16.923202 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e637a00-c249-43a8-8773-af49dd9b823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:16.923973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e637a00-c249-43a8-8773-af49dd9b823c admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:16.924373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e637a00-c249-43a8-8773-af49dd9b823c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:16.924814 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e637a00-c249-43a8-8773-af49dd9b823c admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:16.925681 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e637a00-c249-43a8-8773-af49dd9b823c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:16.927888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e637a00-c249-43a8-8773-af49dd9b823c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:16.966717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b6044-ba06-4f6d-9744-bed96e3a7cd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:16.969302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b6044-ba06-4f6d-9744-bed96e3a7cd7 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:17.010657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 834/1676] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:33:17.044207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 843/1677] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:16 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:17.045122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b6044-ba06-4f6d-9744-bed96e3a7cd7 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:17.061546 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b6044-ba06-4f6d-9744-bed96e3a7cd7 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:17.168968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bd986-82cf-473a-a4d2-a62208fa9cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:17.169975 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bd986-82cf-473a-a4d2-a62208fa9cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:17.170416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bd986-82cf-473a-a4d2-a62208fa9cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:17.170850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9bd986-82cf-473a-a4d2-a62208fa9cd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:17.171665 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9bd986-82cf-473a-a4d2-a62208fa9cd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:17.173767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9bd986-82cf-473a-a4d2-a62208fa9cd1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:17.269800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 835/1678] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:17 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:17.274266 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e637a00-c249-43a8-8773-af49dd9b823c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:17.292718 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e637a00-c249-43a8-8773-af49dd9b823c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:17.296831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd647b7-59be-40c1-9ef7-cb550d59eb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:17.297461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd647b7-59be-40c1-9ef7-cb550d59eb24 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:17.297792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd647b7-59be-40c1-9ef7-cb550d59eb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:17.297966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd647b7-59be-40c1-9ef7-cb550d59eb24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:17.298643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd647b7-59be-40c1-9ef7-cb550d59eb24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:17.300227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd647b7-59be-40c1-9ef7-cb550d59eb24 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:17.311727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 844/1679] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:17.315767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bd986-82cf-473a-a4d2-a62208fa9cd1 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:17.326889 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9bd986-82cf-473a-a4d2-a62208fa9cd1 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:17.346100 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d75f-0361-450d-8d2e-79b96011d056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:17.347159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d75f-0361-450d-8d2e-79b96011d056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:17.347540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d75f-0361-450d-8d2e-79b96011d056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:17.347876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d75f-0361-450d-8d2e-79b96011d056 admin admin] PATH_INFO: `/v3/projects/898020ba548e4fc1b4a83e5b7d047395/users/81d2440c29904f0484b70f5e648bf05e/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:17.368510 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281d75f-0361-450d-8d2e-79b96011d056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=898020ba548e4fc1b4a83e5b7d047395, user_id=81d2440c29904f0484b70f5e648bf05e, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:17.371332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281d75f-0361-450d-8d2e-79b96011d056 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:17.549397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 836/1680] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:17 2022] PUT /identity/v3/projects/898020ba548e4fc1b4a83e5b7d047395/users/81d2440c29904f0484b70f5e648bf05e/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:17.549550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd647b7-59be-40c1-9ef7-cb550d59eb24 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:17.559585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd647b7-59be-40c1-9ef7-cb550d59eb24 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:17.628921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468d2c3-e4d6-4baa-9dca-35dc017060ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:17.629611 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468d2c3-e4d6-4baa-9dca-35dc017060ee admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:17.629897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468d2c3-e4d6-4baa-9dca-35dc017060ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:17.630215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8468d2c3-e4d6-4baa-9dca-35dc017060ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:17.630965 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468d2c3-e4d6-4baa-9dca-35dc017060ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:17.632469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8468d2c3-e4d6-4baa-9dca-35dc017060ee admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:17.650404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 845/1681] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:17.651567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d75f-0361-450d-8d2e-79b96011d056 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:17.672997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d75f-0361-450d-8d2e-79b96011d056 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:17.674368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883da81-3ce7-4a00-9eae-df501431baab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:17.675038 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883da81-3ce7-4a00-9eae-df501431baab admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:17.675311 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883da81-3ce7-4a00-9eae-df501431baab admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:17.675580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883da81-3ce7-4a00-9eae-df501431baab admin admin] PATH_INFO: `/v3/projects/898020ba548e4fc1b4a83e5b7d047395/users/81d2440c29904f0484b70f5e648bf05e/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:17.677457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883da81-3ce7-4a00-9eae-df501431baab admin admin] RBAC: Authorizing `identity:create_grant(project_id=898020ba548e4fc1b4a83e5b7d047395, user_id=81d2440c29904f0484b70f5e648bf05e, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:17.680114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883da81-3ce7-4a00-9eae-df501431baab admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:17.690259 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 837/1682] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:17 2022] PUT /identity/v3/projects/898020ba548e4fc1b4a83e5b7d047395/users/81d2440c29904f0484b70f5e648bf05e/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:18.096229 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468d2c3-e4d6-4baa-9dca-35dc017060ee admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:18.099151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8468d2c3-e4d6-4baa-9dca-35dc017060ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:18.128671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8468d2c3-e4d6-4baa-9dca-35dc017060ee admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:18.205359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:18.206650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:18.207175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:18.207524 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] PATH_INFO: `/v3/users/5b1890fccda44fe6ba42ed3e1a5b3261` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:18.208823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b1890fccda44fe6ba42ed3e1a5b3261)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:18.210930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:18.264004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:18.467376 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b1890fccda44fe6ba42ed3e1a5b3261'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:18.468524 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b1890fccda44fe6ba42ed3e1a5b3261'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:18.501638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b1890fccda44fe6ba42ed3e1a5b3261'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:18.599165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b1890fccda44fe6ba42ed3e1a5b3261'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:18.600029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b1890fccda44fe6ba42ed3e1a5b3261'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:18.621463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 846/1683] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:18 2022] DELETE /identity/v3/users/5b1890fccda44fe6ba42ed3e1a5b3261 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:18.630676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ce91d-5e62-474f-9100-35fc9495d12d None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:18.631428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ce91d-5e62-474f-9100-35fc9495d12d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:18.632184 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ce91d-5e62-474f-9100-35fc9495d12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:18.680691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e09ce91d-5e62-474f-9100-35fc9495d12d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:18.687436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e09ce91d-5e62-474f-9100-35fc9495d12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:18.781741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 838/1684] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:18.816452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:18.818773 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:18.821635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:18.888883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac99ca-4f93-4898-a831-1b41d28d17a0 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:18.892692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb7782-773f-479a-a552-c5d5589f9f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:18.893490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb7782-773f-479a-a552-c5d5589f9f86 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:18.893777 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb7782-773f-479a-a552-c5d5589f9f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:18.894243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb7782-773f-479a-a552-c5d5589f9f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:18.904348 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb7782-773f-479a-a552-c5d5589f9f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:18.905855 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb7782-773f-479a-a552-c5d5589f9f86 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:18.916060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 847/1685] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:19.811233 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ce91d-5e62-474f-9100-35fc9495d12d None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:19.812079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e462057-b9fb-42ca-9a5c-dcdbade2ac56 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:19.812504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e462057-b9fb-42ca-9a5c-dcdbade2ac56 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:19.813139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e462057-b9fb-42ca-9a5c-dcdbade2ac56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:19.813728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e09ce91d-5e62-474f-9100-35fc9495d12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:19.835360 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ce91d-5e62-474f-9100-35fc9495d12d None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:19.846534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:19.847193 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:19.847637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:19.848057 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] PATH_INFO: `/v3/projects/19e166a52dd5412499b10ecb92d1093a` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:19.850256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=19e166a52dd5412499b10ecb92d1093a)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:19.852515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:19.871237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7e462057-b9fb-42ca-9a5c-dcdbade2ac56 None None] MFA Rules not processed for user `81d2440c29904f0484b70f5e648bf05e`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:19.873409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7e462057-b9fb-42ca-9a5c-dcdbade2ac56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:19.949561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 848/1686] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:19.961740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e462057-b9fb-42ca-9a5c-dcdbade2ac56 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:19.964535 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7e462057-b9fb-42ca-9a5c-dcdbade2ac56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:20.001283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19e166a52dd5412499b10ecb92d1093a', 'request_id': 'req-ec466360-969b-4af4-bcbc-93a560aaac5d'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:20.010649 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19e166a52dd5412499b10ecb92d1093a', 'request_id': 'req-ec466360-969b-4af4-bcbc-93a560aaac5d'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:20.034222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e462057-b9fb-42ca-9a5c-dcdbade2ac56 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:20.091452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19e166a52dd5412499b10ecb92d1093a', 'request_id': 'req-ec466360-969b-4af4-bcbc-93a560aaac5d'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:20.103983 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] The token cache is being invalidate because project 19e166a52dd5412499b10ecb92d1093a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:20.104607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:20.105538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:20.108356 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 839/1687] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:19 2022] DELETE /identity/v3/projects/19e166a52dd5412499b10ecb92d1093a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:20.110827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb47538-90d6-4d27-8a99-21b3a68c1d3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:20.111731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb47538-90d6-4d27-8a99-21b3a68c1d3d service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:20.112016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb47538-90d6-4d27-8a99-21b3a68c1d3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:20.112300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb47538-90d6-4d27-8a99-21b3a68c1d3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:20.113042 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:20.114373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-adb47538-90d6-4d27-8a99-21b3a68c1d3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:20.116735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:20.200481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec466360-969b-4af4-bcbc-93a560aaac5d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:20.204323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb47538-90d6-4d27-8a99-21b3a68c1d3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:20.206098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb47538-90d6-4d27-8a99-21b3a68c1d3d service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:20.223832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 849/1688] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:20.229032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:20.230209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:20.230623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:20.231358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] PATH_INFO: `/v3/users/1ca46003975d4aed94a5ef2a1e9020a4` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:20.234208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca46003975d4aed94a5ef2a1e9020a4)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:20.236797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:20.371178 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:20.445118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca46003975d4aed94a5ef2a1e9020a4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:20.446073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca46003975d4aed94a5ef2a1e9020a4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:20.578008 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca46003975d4aed94a5ef2a1e9020a4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:20.578679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca46003975d4aed94a5ef2a1e9020a4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:20.588610 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca46003975d4aed94a5ef2a1e9020a4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:20.630846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 840/1689] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:20 2022] DELETE /identity/v3/users/1ca46003975d4aed94a5ef2a1e9020a4 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:21.177240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177b2e6-6f6f-4c0d-8b1d-e40e16549a22 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:21.180769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177b2e6-6f6f-4c0d-8b1d-e40e16549a22 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:21.181189 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177b2e6-6f6f-4c0d-8b1d-e40e16549a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:21.271169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-6177b2e6-6f6f-4c0d-8b1d-e40e16549a22 None None] MFA Rules not processed for user `22919b0a4b1e4e09b9796a93539547b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:21.273037 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6177b2e6-6f6f-4c0d-8b1d-e40e16549a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:21.372370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 850/1690] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:21.630292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:21.633532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:21.714397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641ada3-9a9b-44ab-9465-dae5c1555b05 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:21.721507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:21.722761 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:21.723044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:21.723329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] PATH_INFO: `/v3/projects/3553bdbadbfa4cbea478779087052022` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:21.726628 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3553bdbadbfa4cbea478779087052022)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:21.728802 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:21.828765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:21.967241 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3553bdbadbfa4cbea478779087052022', 'request_id': 'req-80d48ce7-e30e-4c41-9907-24f36a088fdd'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:21.973981 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3553bdbadbfa4cbea478779087052022', 'request_id': 'req-80d48ce7-e30e-4c41-9907-24f36a088fdd'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:22.137551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3553bdbadbfa4cbea478779087052022', 'request_id': 'req-80d48ce7-e30e-4c41-9907-24f36a088fdd'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:22.150237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] The token cache is being invalidate because project 3553bdbadbfa4cbea478779087052022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:22.150808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:22.151556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-80d48ce7-e30e-4c41-9907-24f36a088fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:22.157243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 841/1691] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:21 2022] DELETE /identity/v3/projects/3553bdbadbfa4cbea478779087052022 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:22.159116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb5e8-88d1-47c7-a122-03e566b64ec2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:22.159810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb5e8-88d1-47c7-a122-03e566b64ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:22.160088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fb5e8-88d1-47c7-a122-03e566b64ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:22.184720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-534fb5e8-88d1-47c7-a122-03e566b64ec2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:22.186220 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-534fb5e8-88d1-47c7-a122-03e566b64ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:22.282244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 851/1692] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:22.283580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ac77c-ea13-41ce-aa65-0767f3d01583 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:22.285067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ac77c-ea13-41ce-aa65-0767f3d01583 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:22.285441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ac77c-ea13-41ce-aa65-0767f3d01583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:22.308880 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-912ac77c-ea13-41ce-aa65-0767f3d01583 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:22.310433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-912ac77c-ea13-41ce-aa65-0767f3d01583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:22.323452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 842/1693] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:22.328164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fb5e8-88d1-47c7-a122-03e566b64ec2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:22.338384 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ac77c-ea13-41ce-aa65-0767f3d01583 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:22.344707 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fb5e8-88d1-47c7-a122-03e566b64ec2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:22.345416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-912ac77c-ea13-41ce-aa65-0767f3d01583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:22.358071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4221ead-4c10-4a40-b7c4-d2cb67318168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:22.359201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4221ead-4c10-4a40-b7c4-d2cb67318168 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:22.359629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4221ead-4c10-4a40-b7c4-d2cb67318168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:22.360051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4221ead-4c10-4a40-b7c4-d2cb67318168 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:22.360949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4221ead-4c10-4a40-b7c4-d2cb67318168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:33:22.361476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4221ead-4c10-4a40-b7c4-d2cb67318168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:22.363251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4221ead-4c10-4a40-b7c4-d2cb67318168 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:22.380487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 852/1694] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:33:22.385208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4221ead-4c10-4a40-b7c4-d2cb67318168 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:22.399165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4221ead-4c10-4a40-b7c4-d2cb67318168 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:22.404791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60041676-280d-433d-8a2a-e2f44f782f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:22.405556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041676-280d-433d-8a2a-e2f44f782f48 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:22.405839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041676-280d-433d-8a2a-e2f44f782f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:22.406140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60041676-280d-433d-8a2a-e2f44f782f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:22.406936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60041676-280d-433d-8a2a-e2f44f782f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:22.408866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60041676-280d-433d-8a2a-e2f44f782f48 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:22.429191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ac77c-ea13-41ce-aa65-0767f3d01583 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:22.437977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63834d3-63e5-4a8d-a48c-9e91d3725d70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:22.437977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63834d3-63e5-4a8d-a48c-9e91d3725d70 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:22.437977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63834d3-63e5-4a8d-a48c-9e91d3725d70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:22.437977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63834d3-63e5-4a8d-a48c-9e91d3725d70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:22.438305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b63834d3-63e5-4a8d-a48c-9e91d3725d70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:22.502581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63834d3-63e5-4a8d-a48c-9e91d3725d70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:22.504015 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63834d3-63e5-4a8d-a48c-9e91d3725d70 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:22.514034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 843/1695] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:33:22.584509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 853/1696] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:22 2022] POST /identity/v3/projects => generated 399 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:22.588800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63834d3-63e5-4a8d-a48c-9e91d3725d70 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:22.601370 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63834d3-63e5-4a8d-a48c-9e91d3725d70 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:22.663885 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b4294-317c-40d5-9da2-618d6831e07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:22.664554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b4294-317c-40d5-9da2-618d6831e07a admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:22.664748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b4294-317c-40d5-9da2-618d6831e07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:22.665228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b4294-317c-40d5-9da2-618d6831e07a admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:22.665883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b4294-317c-40d5-9da2-618d6831e07a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:22.667828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b4294-317c-40d5-9da2-618d6831e07a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:22.773520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 844/1697] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:22 2022] POST /identity/v3/users => generated 300 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:22.781309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60041676-280d-433d-8a2a-e2f44f782f48 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:22.811433 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60041676-280d-433d-8a2a-e2f44f782f48 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:22.819790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4254279-a13d-41eb-b16e-dbd0198e8de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:22.820750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4254279-a13d-41eb-b16e-dbd0198e8de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:22.820750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4254279-a13d-41eb-b16e-dbd0198e8de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:22.821087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4254279-a13d-41eb-b16e-dbd0198e8de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:22.821816 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4254279-a13d-41eb-b16e-dbd0198e8de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:22.824397 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4254279-a13d-41eb-b16e-dbd0198e8de1 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:22.841544 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 854/1698] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:22.848293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b4294-317c-40d5-9da2-618d6831e07a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:22.859303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b4294-317c-40d5-9da2-618d6831e07a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:22.864050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae864b-7e70-4f18-ad3b-3221287c5c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:22.864653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae864b-7e70-4f18-ad3b-3221287c5c0c admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:22.864929 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae864b-7e70-4f18-ad3b-3221287c5c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:22.865203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae864b-7e70-4f18-ad3b-3221287c5c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:22.865855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae864b-7e70-4f18-ad3b-3221287c5c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:22.867521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae864b-7e70-4f18-ad3b-3221287c5c0c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:22.878576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 845/1699] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:22.882984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4254279-a13d-41eb-b16e-dbd0198e8de1 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:22.892791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4254279-a13d-41eb-b16e-dbd0198e8de1 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:22.904356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aa255-2a08-472f-ac2e-09ef5cde2271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:22.905102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa255-2a08-472f-ac2e-09ef5cde2271 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:22.905385 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa255-2a08-472f-ac2e-09ef5cde2271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:22.905663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa255-2a08-472f-ac2e-09ef5cde2271 admin admin] PATH_INFO: `/v3/projects/664e61e9122f40f8b60b42597d957ba8/users/7d2ae922f11048cea108f7dab5d05d24/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:22.920747 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aa255-2a08-472f-ac2e-09ef5cde2271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=664e61e9122f40f8b60b42597d957ba8, user_id=7d2ae922f11048cea108f7dab5d05d24, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:22.923459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aa255-2a08-472f-ac2e-09ef5cde2271 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:23.045564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 855/1700] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:22 2022] PUT /identity/v3/projects/664e61e9122f40f8b60b42597d957ba8/users/7d2ae922f11048cea108f7dab5d05d24/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:23.055843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1bab8-fce4-4122-bffb-6fade4d2a637 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:23.056160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1bab8-fce4-4122-bffb-6fade4d2a637 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:23.056432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1bab8-fce4-4122-bffb-6fade4d2a637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:23.090620 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-19b1bab8-fce4-4122-bffb-6fade4d2a637 None None] MFA Rules not processed for user `7d2ae922f11048cea108f7dab5d05d24`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:23.092102 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-19b1bab8-fce4-4122-bffb-6fade4d2a637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:23.167221 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 846/1701] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:23 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:23.178826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aa255-2a08-472f-ac2e-09ef5cde2271 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:23.248033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aa255-2a08-472f-ac2e-09ef5cde2271 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:23.254390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a297c4a-0ced-473d-a801-423a51976c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:23.254390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297c4a-0ced-473d-a801-423a51976c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:23.254390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297c4a-0ced-473d-a801-423a51976c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:23.254763 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297c4a-0ced-473d-a801-423a51976c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:23.255690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a297c4a-0ced-473d-a801-423a51976c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:23.257866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a297c4a-0ced-473d-a801-423a51976c75 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:23.262527 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:23.383758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 856/1702] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:23 2022] POST /identity/v3/projects => generated 401 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:23.387867 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1bab8-fce4-4122-bffb-6fade4d2a637 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:23.399354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1bab8-fce4-4122-bffb-6fade4d2a637 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:23.459690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee29c6-8276-4996-a10a-339606aeda3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:23.460301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee29c6-8276-4996-a10a-339606aeda3a admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:23.460580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee29c6-8276-4996-a10a-339606aeda3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:23.460851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee29c6-8276-4996-a10a-339606aeda3a admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:23.461493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee29c6-8276-4996-a10a-339606aeda3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:23.463431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee29c6-8276-4996-a10a-339606aeda3a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:23.569298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 847/1703] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:23 2022] POST /identity/v3/users => generated 307 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:23.570679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a297c4a-0ced-473d-a801-423a51976c75 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:23.581856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a297c4a-0ced-473d-a801-423a51976c75 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:23.588230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3278b0c-33af-4fd8-930f-511d17c0e2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:23.588983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3278b0c-33af-4fd8-930f-511d17c0e2ef admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:23.589326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3278b0c-33af-4fd8-930f-511d17c0e2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:23.589693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3278b0c-33af-4fd8-930f-511d17c0e2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:23.590527 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3278b0c-33af-4fd8-930f-511d17c0e2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:23.592267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3278b0c-33af-4fd8-930f-511d17c0e2ef admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:23.608480 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 857/1704] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:23.609973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee29c6-8276-4996-a10a-339606aeda3a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:23.619813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee29c6-8276-4996-a10a-339606aeda3a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:23.626198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47b601-ddeb-4a40-ae82-63f62f6e39f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:23.627970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b601-ddeb-4a40-ae82-63f62f6e39f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:23.627970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b601-ddeb-4a40-ae82-63f62f6e39f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:23.628141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47b601-ddeb-4a40-ae82-63f62f6e39f7 admin admin] PATH_INFO: `/v3/projects/7dc5673ea198476a9bdad035b5b3944e/users/fb83dd235c2b4fe38c54d418c3b55990/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:23.643539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47b601-ddeb-4a40-ae82-63f62f6e39f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc5673ea198476a9bdad035b5b3944e, user_id=fb83dd235c2b4fe38c54d418c3b55990, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:23.646248 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47b601-ddeb-4a40-ae82-63f62f6e39f7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:23.759720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 848/1705] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:23 2022] PUT /identity/v3/projects/7dc5673ea198476a9bdad035b5b3944e/users/fb83dd235c2b4fe38c54d418c3b55990/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:23.763807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3278b0c-33af-4fd8-930f-511d17c0e2ef admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:23.774251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3278b0c-33af-4fd8-930f-511d17c0e2ef admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:23.829023 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d4bdf-6788-4be6-83f3-b0c187ac6e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:23.829625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d4bdf-6788-4be6-83f3-b0c187ac6e18 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:23.829905 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d4bdf-6788-4be6-83f3-b0c187ac6e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:23.830211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d4bdf-6788-4be6-83f3-b0c187ac6e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:23.830851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d4bdf-6788-4be6-83f3-b0c187ac6e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:23.832483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d4bdf-6788-4be6-83f3-b0c187ac6e18 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:23.841937 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 858/1706] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:23.846500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47b601-ddeb-4a40-ae82-63f62f6e39f7 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:23.857707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47b601-ddeb-4a40-ae82-63f62f6e39f7 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:23.862306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898d81d-bff4-483c-88b0-5e2cafd8bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:23.862942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d81d-bff4-483c-88b0-5e2cafd8bd5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:23.863217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d81d-bff4-483c-88b0-5e2cafd8bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:23.863486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d81d-bff4-483c-88b0-5e2cafd8bd5f admin admin] PATH_INFO: `/v3/projects/7dc5673ea198476a9bdad035b5b3944e/users/fb83dd235c2b4fe38c54d418c3b55990/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:23.865520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898d81d-bff4-483c-88b0-5e2cafd8bd5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc5673ea198476a9bdad035b5b3944e, user_id=fb83dd235c2b4fe38c54d418c3b55990, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:23.868245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898d81d-bff4-483c-88b0-5e2cafd8bd5f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:23.879562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 849/1707] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:23 2022] PUT /identity/v3/projects/7dc5673ea198476a9bdad035b5b3944e/users/fb83dd235c2b4fe38c54d418c3b55990/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:23.891032 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8686ee2-8a87-4ba3-b123-7b534bd8a4a2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:23.891439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8686ee2-8a87-4ba3-b123-7b534bd8a4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:23.891663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8686ee2-8a87-4ba3-b123-7b534bd8a4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:23.921830 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-e8686ee2-8a87-4ba3-b123-7b534bd8a4a2 None None] MFA Rules not processed for user `fb83dd235c2b4fe38c54d418c3b55990`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:23.923092 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e8686ee2-8a87-4ba3-b123-7b534bd8a4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:23.991798 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 859/1708] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:23 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:24.003091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898d81d-bff4-483c-88b0-5e2cafd8bd5f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:24.004819 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e898d81d-bff4-483c-88b0-5e2cafd8bd5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:24.062206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898d81d-bff4-483c-88b0-5e2cafd8bd5f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:24.065876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b7469-f357-4f32-b7ca-5fc4a8f8f182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:24.066505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b7469-f357-4f32-b7ca-5fc4a8f8f182 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:24.066777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b7469-f357-4f32-b7ca-5fc4a8f8f182 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:24.067047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b7469-f357-4f32-b7ca-5fc4a8f8f182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:24.076244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b7469-f357-4f32-b7ca-5fc4a8f8f182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:24.077794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b7469-f357-4f32-b7ca-5fc4a8f8f182 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:24.086912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 850/1709] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:24 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:24.350227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8686ee2-8a87-4ba3-b123-7b534bd8a4a2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:24.360561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8686ee2-8a87-4ba3-b123-7b534bd8a4a2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:24.364182 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d0480-7183-4970-95de-5d70e1e92fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:24.364784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0480-7183-4970-95de-5d70e1e92fbd service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:24.365056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0480-7183-4970-95de-5d70e1e92fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:24.365326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d0480-7183-4970-95de-5d70e1e92fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:24.428496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d0480-7183-4970-95de-5d70e1e92fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:24.430129 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d0480-7183-4970-95de-5d70e1e92fbd service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:24.439394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 860/1710] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:24 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:24.508009 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:24.782411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:26.096624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:26.188872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:26.810293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:27.119835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:27.633897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:28.280655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:28.680654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:28.845817 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:29.451582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:29.577625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:30.344606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:30.445505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:30.914149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:30.988131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:31.222560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:31.387593 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:31.468838 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:31.719839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:31.776636 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:31.965342 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:31.999627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:32.096346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:32.182100 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:32.205153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:32.331151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:32.353998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:32.434108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:32.502906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:32.531268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:32.699125 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:33.548197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:34.547545 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b7469-f357-4f32-b7ca-5fc4a8f8f182 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:34.613561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b7469-f357-4f32-b7ca-5fc4a8f8f182 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:34.618407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:34.619154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:34.619465 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:34.619741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] PATH_INFO: `/v3/users/7d2ae922f11048cea108f7dab5d05d24` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:34.621042 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d2ae922f11048cea108f7dab5d05d24)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:34.623044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:34.853389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d2ae922f11048cea108f7dab5d05d24'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:34.854138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d2ae922f11048cea108f7dab5d05d24'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:34.873827 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:34.942102 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d2ae922f11048cea108f7dab5d05d24'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:34.942809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d2ae922f11048cea108f7dab5d05d24'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:34.951174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d2ae922f11048cea108f7dab5d05d24'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:34.969016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 851/1711] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:34 2022] DELETE /identity/v3/users/7d2ae922f11048cea108f7dab5d05d24 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:34.974160 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2141d6e-893d-4e97-8845-9c0b7d813df9 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:34.974160 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2141d6e-893d-4e97-8845-9c0b7d813df9 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:34.974160 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2141d6e-893d-4e97-8845-9c0b7d813df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:34.997104 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c2141d6e-893d-4e97-8845-9c0b7d813df9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:34.998356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c2141d6e-893d-4e97-8845-9c0b7d813df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:35.061884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 861/1712] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:35.069373 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:35.071012 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:35.134315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d2fae-eb0b-43c1-b08e-c2adeed63f03 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:35.137874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb7e97-c601-4e10-ba48-1fdacb71fcbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:35.139038 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb7e97-c601-4e10-ba48-1fdacb71fcbc service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:35.139315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb7e97-c601-4e10-ba48-1fdacb71fcbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:35.139581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb7e97-c601-4e10-ba48-1fdacb71fcbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:35.148546 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb7e97-c601-4e10-ba48-1fdacb71fcbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:35.149954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb7e97-c601-4e10-ba48-1fdacb71fcbc service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:35.160836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 852/1713] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:35.683558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2141d6e-893d-4e97-8845-9c0b7d813df9 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:35.687234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c2141d6e-893d-4e97-8845-9c0b7d813df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:35.702617 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2141d6e-893d-4e97-8845-9c0b7d813df9 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:35.708656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:35.709291 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:35.709564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:35.709831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] PATH_INFO: `/v3/projects/664e61e9122f40f8b60b42597d957ba8` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:35.710988 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=664e61e9122f40f8b60b42597d957ba8)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:35.714356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:35.862555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '664e61e9122f40f8b60b42597d957ba8', 'request_id': 'req-1eeaaffa-a135-4a88-b89e-49335af46a6e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:35.869132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '664e61e9122f40f8b60b42597d957ba8', 'request_id': 'req-1eeaaffa-a135-4a88-b89e-49335af46a6e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:36.005182 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '664e61e9122f40f8b60b42597d957ba8', 'request_id': 'req-1eeaaffa-a135-4a88-b89e-49335af46a6e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:36.036397 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] The token cache is being invalidate because project 664e61e9122f40f8b60b42597d957ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:36.044158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:36.044158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:36.044158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 862/1714] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:35 2022] DELETE /identity/v3/projects/664e61e9122f40f8b60b42597d957ba8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:36.046601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb7e97-c601-4e10-ba48-1fdacb71fcbc service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:36.058506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb7e97-c601-4e10-ba48-1fdacb71fcbc service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:36.125325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b4898-1eef-40a7-af6e-effb6430152d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:36.126198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b4898-1eef-40a7-af6e-effb6430152d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:36.126575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b4898-1eef-40a7-af6e-effb6430152d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:36.127005 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b4898-1eef-40a7-af6e-effb6430152d admin admin] PATH_INFO: `/v3/users/fb83dd235c2b4fe38c54d418c3b55990` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:36.128652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b4898-1eef-40a7-af6e-effb6430152d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb83dd235c2b4fe38c54d418c3b55990)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:36.130828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b4898-1eef-40a7-af6e-effb6430152d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:36.150905 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:36.459757 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-782b4898-1eef-40a7-af6e-effb6430152d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb83dd235c2b4fe38c54d418c3b55990'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:36.460745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-782b4898-1eef-40a7-af6e-effb6430152d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb83dd235c2b4fe38c54d418c3b55990'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:36.565344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-782b4898-1eef-40a7-af6e-effb6430152d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb83dd235c2b4fe38c54d418c3b55990'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:36.567724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-782b4898-1eef-40a7-af6e-effb6430152d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb83dd235c2b4fe38c54d418c3b55990'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:36.573978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-782b4898-1eef-40a7-af6e-effb6430152d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb83dd235c2b4fe38c54d418c3b55990'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:36.591696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 853/1715] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:36 2022] DELETE /identity/v3/users/fb83dd235c2b4fe38c54d418c3b55990 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:37.000244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:37.141380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:37.142715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:37.155423 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaaffa-a135-4a88-b89e-49335af46a6e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:37.184630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:37.211628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:37.212250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:37.212535 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:37.212806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] PATH_INFO: `/v3/projects/7dc5673ea198476a9bdad035b5b3944e` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:37.213923 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dc5673ea198476a9bdad035b5b3944e)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:37.215855 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:37.337203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc5673ea198476a9bdad035b5b3944e', 'request_id': 'req-e8231130-9036-4e32-b292-8ad3d5895294'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:37.343478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc5673ea198476a9bdad035b5b3944e', 'request_id': 'req-e8231130-9036-4e32-b292-8ad3d5895294'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:37.422621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc5673ea198476a9bdad035b5b3944e', 'request_id': 'req-e8231130-9036-4e32-b292-8ad3d5895294'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:37.433526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] The token cache is being invalidate because project 7dc5673ea198476a9bdad035b5b3944e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:37.433815 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:37.434462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e8231130-9036-4e32-b292-8ad3d5895294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:37.440737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 863/1716] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:37 2022] DELETE /identity/v3/projects/7dc5673ea198476a9bdad035b5b3944e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:37.441264 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffe5a-423a-455e-b556-cd1252ec28be None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:37.441711 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffe5a-423a-455e-b556-cd1252ec28be None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:37.441805 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabffe5a-423a-455e-b556-cd1252ec28be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:37.461052 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-cabffe5a-423a-455e-b556-cd1252ec28be None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:37.463745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cabffe5a-423a-455e-b556-cd1252ec28be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:37.521515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 854/1717] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:37.529831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475ab63-93fe-4683-b860-7a2a18c91a18 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:37.530225 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475ab63-93fe-4683-b860-7a2a18c91a18 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:37.530588 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475ab63-93fe-4683-b860-7a2a18c91a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:37.553714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-d475ab63-93fe-4683-b860-7a2a18c91a18 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:37.554946 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d475ab63-93fe-4683-b860-7a2a18c91a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:37.563003 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 864/1718] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:37.567062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabffe5a-423a-455e-b556-cd1252ec28be None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:37.576664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabffe5a-423a-455e-b556-cd1252ec28be None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:37.581204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da6a33-0f40-4dcf-8bee-d869f482642a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:37.581809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da6a33-0f40-4dcf-8bee-d869f482642a admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:37.582120 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da6a33-0f40-4dcf-8bee-d869f482642a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:37.582394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da6a33-0f40-4dcf-8bee-d869f482642a admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:37.583042 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da6a33-0f40-4dcf-8bee-d869f482642a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:33:37.583345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da6a33-0f40-4dcf-8bee-d869f482642a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:37.584931 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da6a33-0f40-4dcf-8bee-d869f482642a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:37.597109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 855/1719] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:33:37.601463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475ab63-93fe-4683-b860-7a2a18c91a18 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:37.610801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475ab63-93fe-4683-b860-7a2a18c91a18 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:37.615463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392c9f7-94d4-435c-bb27-aa4c8b870dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:37.616066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392c9f7-94d4-435c-bb27-aa4c8b870dab admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:37.616342 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392c9f7-94d4-435c-bb27-aa4c8b870dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:37.616609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392c9f7-94d4-435c-bb27-aa4c8b870dab admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:37.617244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392c9f7-94d4-435c-bb27-aa4c8b870dab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:37.619178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392c9f7-94d4-435c-bb27-aa4c8b870dab admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:37.720854 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 865/1720] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:37 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:37.724864 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da6a33-0f40-4dcf-8bee-d869f482642a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:37.734853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da6a33-0f40-4dcf-8bee-d869f482642a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:37.790742 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef5a74-5d3d-4ed0-9a18-d32859f63b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:37.791377 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef5a74-5d3d-4ed0-9a18-d32859f63b61 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:37.791663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef5a74-5d3d-4ed0-9a18-d32859f63b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:37.791938 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef5a74-5d3d-4ed0-9a18-d32859f63b61 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:37.792581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef5a74-5d3d-4ed0-9a18-d32859f63b61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:37.794541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef5a74-5d3d-4ed0-9a18-d32859f63b61 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:37.885572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 856/1721] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:37 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:37.913291 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392c9f7-94d4-435c-bb27-aa4c8b870dab admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:37.924419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392c9f7-94d4-435c-bb27-aa4c8b870dab admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:37.929073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0137ab-a4d6-49df-ac3e-e8091a0e602a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:37.929679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0137ab-a4d6-49df-ac3e-e8091a0e602a admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:37.929961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0137ab-a4d6-49df-ac3e-e8091a0e602a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:37.930267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0137ab-a4d6-49df-ac3e-e8091a0e602a admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:37.930909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0137ab-a4d6-49df-ac3e-e8091a0e602a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:37.932490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0137ab-a4d6-49df-ac3e-e8091a0e602a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:37.946002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 866/1722] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:37.950756 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef5a74-5d3d-4ed0-9a18-d32859f63b61 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:37.957750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef5a74-5d3d-4ed0-9a18-d32859f63b61 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:37.962641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cf626-be17-4760-862f-a6547ee5a552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:37.963364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cf626-be17-4760-862f-a6547ee5a552 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:37.963648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cf626-be17-4760-862f-a6547ee5a552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:37.963915 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cf626-be17-4760-862f-a6547ee5a552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:37.964590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cf626-be17-4760-862f-a6547ee5a552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:37.966225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cf626-be17-4760-862f-a6547ee5a552 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:37.978721 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 857/1723] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:37.983613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0137ab-a4d6-49df-ac3e-e8091a0e602a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:37.993655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0137ab-a4d6-49df-ac3e-e8091a0e602a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:37.998926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8c7a6-78f9-49f2-8885-af4c0e94647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:37.999581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c7a6-78f9-49f2-8885-af4c0e94647a admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:37.999861 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c7a6-78f9-49f2-8885-af4c0e94647a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:38.000135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8c7a6-78f9-49f2-8885-af4c0e94647a admin admin] PATH_INFO: `/v3/projects/f6a37ae65eb34b5cb31549a8864fe86d/users/2b17a1ed8c0749189cede37691ba07a2/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:38.015178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8c7a6-78f9-49f2-8885-af4c0e94647a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a37ae65eb34b5cb31549a8864fe86d, user_id=2b17a1ed8c0749189cede37691ba07a2, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:38.019484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8c7a6-78f9-49f2-8885-af4c0e94647a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:38.068531 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:38.137165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 867/1724] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:37 2022] PUT /identity/v3/projects/f6a37ae65eb34b5cb31549a8864fe86d/users/2b17a1ed8c0749189cede37691ba07a2/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:38.148707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef3171-898d-462e-98ae-1dfc666a5885 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:38.148990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef3171-898d-462e-98ae-1dfc666a5885 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:38.149260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef3171-898d-462e-98ae-1dfc666a5885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:38.178127 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-a1ef3171-898d-462e-98ae-1dfc666a5885 None None] MFA Rules not processed for user `2b17a1ed8c0749189cede37691ba07a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:38.179321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a1ef3171-898d-462e-98ae-1dfc666a5885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:38.241486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 858/1725] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:38.252454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8c7a6-78f9-49f2-8885-af4c0e94647a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:38.254480 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-90e8c7a6-78f9-49f2-8885-af4c0e94647a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:38.319342 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:38.333339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8c7a6-78f9-49f2-8885-af4c0e94647a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:38.337257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10078a4d-822f-4fb3-adb1-0d3b25f70284 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286cfcbb8334421f87c0d426aa52a30a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:38.341851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10078a4d-822f-4fb3-adb1-0d3b25f70284 service glance] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:38.341851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10078a4d-822f-4fb3-adb1-0d3b25f70284 service glance] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:38.341851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10078a4d-822f-4fb3-adb1-0d3b25f70284 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:38.347459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10078a4d-822f-4fb3-adb1-0d3b25f70284 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:38.349087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10078a4d-822f-4fb3-adb1-0d3b25f70284 service glance] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:38.358729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 868/1726] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:44.998400 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef3171-898d-462e-98ae-1dfc666a5885 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:44.999989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a1ef3171-898d-462e-98ae-1dfc666a5885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:45.011911 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:45.074532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef3171-898d-462e-98ae-1dfc666a5885 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:45.079278 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:45.079992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:45.080337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:45.080825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] PATH_INFO: `/v3/users/34df87739a2f4de0a8b290df6a524ec2` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:45.082106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34df87739a2f4de0a8b290df6a524ec2)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:45.084126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:45.364405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34df87739a2f4de0a8b290df6a524ec2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:45.365098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34df87739a2f4de0a8b290df6a524ec2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:45.456357 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34df87739a2f4de0a8b290df6a524ec2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:45.457049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34df87739a2f4de0a8b290df6a524ec2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:45.469009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34df87739a2f4de0a8b290df6a524ec2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:45.487157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 859/1727] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:44 2022] DELETE /identity/v3/users/34df87739a2f4de0a8b290df6a524ec2 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:45.492137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674d7d-1d4a-4b15-94fd-1bf8d69f6995 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:45.492599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674d7d-1d4a-4b15-94fd-1bf8d69f6995 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:45.492742 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674d7d-1d4a-4b15-94fd-1bf8d69f6995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:45.514504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-10674d7d-1d4a-4b15-94fd-1bf8d69f6995 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:45.515691 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-10674d7d-1d4a-4b15-94fd-1bf8d69f6995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:45.621006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 869/1728] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:45.637022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:45.642616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:45.732242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4edb5-c3ce-42f8-8f81-7dfc84df3da2 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:45.736913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96898bcd-07ad-40f0-8bb2-838e7662f836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:45.737534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96898bcd-07ad-40f0-8bb2-838e7662f836 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:45.737816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96898bcd-07ad-40f0-8bb2-838e7662f836 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:45.738121 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96898bcd-07ad-40f0-8bb2-838e7662f836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:45.749570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96898bcd-07ad-40f0-8bb2-838e7662f836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:45.751124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96898bcd-07ad-40f0-8bb2-838e7662f836 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:45.762063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 860/1729] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:46.264278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10674d7d-1d4a-4b15-94fd-1bf8d69f6995 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:46.265836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-10674d7d-1d4a-4b15-94fd-1bf8d69f6995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:46.281557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10674d7d-1d4a-4b15-94fd-1bf8d69f6995 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:46.286491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:46.287204 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:46.287552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:46.287933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] PATH_INFO: `/v3/projects/7c3d7d79d1d3425ebf3d90d4c5006471` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:46.289137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c3d7d79d1d3425ebf3d90d4c5006471)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:46.292261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:46.404833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c3d7d79d1d3425ebf3d90d4c5006471', 'request_id': 'req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:46.411170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c3d7d79d1d3425ebf3d90d4c5006471', 'request_id': 'req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:46.484961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c3d7d79d1d3425ebf3d90d4c5006471', 'request_id': 'req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:46.495955 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] The token cache is being invalidate because project 7c3d7d79d1d3425ebf3d90d4c5006471 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:46.496527 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:46.496841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:46.498391 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 870/1730] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:46 2022] DELETE /identity/v3/projects/7c3d7d79d1d3425ebf3d90d4c5006471 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:46.502787 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96898bcd-07ad-40f0-8bb2-838e7662f836 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:46.512465 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96898bcd-07ad-40f0-8bb2-838e7662f836 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:46.542273 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:46.572928 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee218c-fcd0-4455-9e2e-73b8d690267c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:46.573569 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee218c-fcd0-4455-9e2e-73b8d690267c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:46.573846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee218c-fcd0-4455-9e2e-73b8d690267c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:46.574169 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee218c-fcd0-4455-9e2e-73b8d690267c admin admin] PATH_INFO: `/v3/users/6c6b9471db8646eca02ea686ed2d47bf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:46.575743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee218c-fcd0-4455-9e2e-73b8d690267c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c6b9471db8646eca02ea686ed2d47bf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:46.577500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee218c-fcd0-4455-9e2e-73b8d690267c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:46.809965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-beee218c-fcd0-4455-9e2e-73b8d690267c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c6b9471db8646eca02ea686ed2d47bf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:46.810956 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-beee218c-fcd0-4455-9e2e-73b8d690267c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c6b9471db8646eca02ea686ed2d47bf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:46.939422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-beee218c-fcd0-4455-9e2e-73b8d690267c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c6b9471db8646eca02ea686ed2d47bf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:46.940122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-beee218c-fcd0-4455-9e2e-73b8d690267c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c6b9471db8646eca02ea686ed2d47bf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:46.948069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-beee218c-fcd0-4455-9e2e-73b8d690267c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c6b9471db8646eca02ea686ed2d47bf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:46.974018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 861/1731] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:46 2022] DELETE /identity/v3/users/6c6b9471db8646eca02ea686ed2d47bf => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:47.589932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:47.597621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:47.608909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fe0ce2-0c1d-4db9-aa03-79f9cd0b1fe1 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:47.686490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:47.687176 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:47.688375 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:47.689140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] PATH_INFO: `/v3/projects/c3a6a5951fc0429b927cfbee2ed35c4c` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:47.691913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a6a5951fc0429b927cfbee2ed35c4c)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:47.694625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:47.702357 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:47.892245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a6a5951fc0429b927cfbee2ed35c4c', 'request_id': 'req-a1da635f-791c-44e4-9de6-90439d72a07c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:47.898424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a6a5951fc0429b927cfbee2ed35c4c', 'request_id': 'req-a1da635f-791c-44e4-9de6-90439d72a07c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:48.103096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a6a5951fc0429b927cfbee2ed35c4c', 'request_id': 'req-a1da635f-791c-44e4-9de6-90439d72a07c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:48.126421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] The token cache is being invalidate because project c3a6a5951fc0429b927cfbee2ed35c4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:48.127216 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:48.128184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a1da635f-791c-44e4-9de6-90439d72a07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:48.129836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 871/1732] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:47 2022] DELETE /identity/v3/projects/c3a6a5951fc0429b927cfbee2ed35c4c => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:48.134840 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986ddb9-da74-4c3e-b96c-5f2f1a364eba None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:48.135264 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986ddb9-da74-4c3e-b96c-5f2f1a364eba None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:48.135397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986ddb9-da74-4c3e-b96c-5f2f1a364eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:48.179456 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-9986ddb9-da74-4c3e-b96c-5f2f1a364eba None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:48.180849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9986ddb9-da74-4c3e-b96c-5f2f1a364eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:48.267105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 862/1733] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:48.276765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f446-4b5f-4fea-afd3-2b43ead6e4fb None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:48.277595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f446-4b5f-4fea-afd3-2b43ead6e4fb None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:48.277595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563f446-4b5f-4fea-afd3-2b43ead6e4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:48.308945 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-1563f446-4b5f-4fea-afd3-2b43ead6e4fb None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:48.310541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1563f446-4b5f-4fea-afd3-2b43ead6e4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:48.319072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 872/1734] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:48.323813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986ddb9-da74-4c3e-b96c-5f2f1a364eba None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:48.336974 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986ddb9-da74-4c3e-b96c-5f2f1a364eba None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:48.345807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a02eb-dc04-4c41-a534-2b028ce70e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:48.347234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a02eb-dc04-4c41-a534-2b028ce70e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:48.347504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a02eb-dc04-4c41-a534-2b028ce70e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:48.347877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a02eb-dc04-4c41-a534-2b028ce70e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:48.348642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a02eb-dc04-4c41-a534-2b028ce70e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:33:48.349197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a02eb-dc04-4c41-a534-2b028ce70e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:48.351205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a02eb-dc04-4c41-a534-2b028ce70e03 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:48.383834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 863/1735] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:33:48.385364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563f446-4b5f-4fea-afd3-2b43ead6e4fb None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:48.404744 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563f446-4b5f-4fea-afd3-2b43ead6e4fb None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:48.414807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa0a78-df11-4dcb-b90c-2f71b49547c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:48.416138 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa0a78-df11-4dcb-b90c-2f71b49547c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:48.416553 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa0a78-df11-4dcb-b90c-2f71b49547c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:48.416977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa0a78-df11-4dcb-b90c-2f71b49547c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:48.418349 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa0a78-df11-4dcb-b90c-2f71b49547c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:48.421949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa0a78-df11-4dcb-b90c-2f71b49547c5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:48.527827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 873/1736] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:48 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:48.532311 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a02eb-dc04-4c41-a534-2b028ce70e03 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:48.542803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a02eb-dc04-4c41-a534-2b028ce70e03 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:48.579782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:48.603388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0613c5-49e6-49c8-afaf-75d1c6996118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:48.604460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0613c5-49e6-49c8-afaf-75d1c6996118 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:48.604460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0613c5-49e6-49c8-afaf-75d1c6996118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:48.604727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0613c5-49e6-49c8-afaf-75d1c6996118 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:48.605519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0613c5-49e6-49c8-afaf-75d1c6996118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:48.607967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0613c5-49e6-49c8-afaf-75d1c6996118 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:48.621909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa0a78-df11-4dcb-b90c-2f71b49547c5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:48.624492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-fffa0a78-df11-4dcb-b90c-2f71b49547c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:48.640402 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa0a78-df11-4dcb-b90c-2f71b49547c5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:48.645581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:48.646478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:48.646915 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:48.647341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] PATH_INFO: `/v3/users/cfa9afff01b3432790a6b63f72a45c96` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:48.648782 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa9afff01b3432790a6b63f72a45c96)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:48.651278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:48.702415 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 864/1737] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:48 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:33:48.706585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0613c5-49e6-49c8-afaf-75d1c6996118 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:48.717238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0613c5-49e6-49c8-afaf-75d1c6996118 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:48.722971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391556c5-2631-4df0-a6e8-ac781ce0b0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:48.723699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391556c5-2631-4df0-a6e8-ac781ce0b0fc admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:48.724089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391556c5-2631-4df0-a6e8-ac781ce0b0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:48.724732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391556c5-2631-4df0-a6e8-ac781ce0b0fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:48.726175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391556c5-2631-4df0-a6e8-ac781ce0b0fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:48.728788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391556c5-2631-4df0-a6e8-ac781ce0b0fc admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:48.743507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 865/1738] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:33:48.744408 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391556c5-2631-4df0-a6e8-ac781ce0b0fc admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:48.756133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391556c5-2631-4df0-a6e8-ac781ce0b0fc admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:48.761748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46006237-b7f5-41be-93b0-0c37061f4e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:48.762391 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006237-b7f5-41be-93b0-0c37061f4e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:48.762674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006237-b7f5-41be-93b0-0c37061f4e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:48.763005 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46006237-b7f5-41be-93b0-0c37061f4e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:48.764134 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46006237-b7f5-41be-93b0-0c37061f4e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:48.765357 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46006237-b7f5-41be-93b0-0c37061f4e36 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:48.775664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 866/1739] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:48.780296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46006237-b7f5-41be-93b0-0c37061f4e36 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:48.789768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46006237-b7f5-41be-93b0-0c37061f4e36 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:48.794491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe8e86-2f90-4a19-865a-5a8a15e3f6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:48.795170 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8e86-2f90-4a19-865a-5a8a15e3f6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:48.795427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8e86-2f90-4a19-865a-5a8a15e3f6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:48.795725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe8e86-2f90-4a19-865a-5a8a15e3f6e5 admin admin] PATH_INFO: `/v3/projects/50d06218d5d44aa0826bf22927331bd0/users/20af20cdaa244bd4955338fbf3c05680/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:48.812267 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe8e86-2f90-4a19-865a-5a8a15e3f6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d06218d5d44aa0826bf22927331bd0, user_id=20af20cdaa244bd4955338fbf3c05680, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:48.815075 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe8e86-2f90-4a19-865a-5a8a15e3f6e5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:48.916103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa9afff01b3432790a6b63f72a45c96'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:48.917036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa9afff01b3432790a6b63f72a45c96'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:48.932391 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa9afff01b3432790a6b63f72a45c96'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:48.970799 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 867/1740] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:48 2022] PUT /identity/v3/projects/50d06218d5d44aa0826bf22927331bd0/users/20af20cdaa244bd4955338fbf3c05680/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:48.981106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae817a7-ecc0-4561-b287-be09f5bf2496 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:48.981389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae817a7-ecc0-4561-b287-be09f5bf2496 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:48.981694 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae817a7-ecc0-4561-b287-be09f5bf2496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:49.019800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-eae817a7-ecc0-4561-b287-be09f5bf2496 None None] MFA Rules not processed for user `20af20cdaa244bd4955338fbf3c05680`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:49.021153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-eae817a7-ecc0-4561-b287-be09f5bf2496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:49.050465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa9afff01b3432790a6b63f72a45c96'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:49.051097 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa9afff01b3432790a6b63f72a45c96'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:49.082682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 874/1741] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:48 2022] DELETE /identity/v3/users/cfa9afff01b3432790a6b63f72a45c96 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:49.104132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 868/1742] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:49.118374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:49.121831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:49.189226 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:49.206347 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab04d38-8257-4546-816c-7d0cc5689664 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:49.210658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b151e5-c8f6-407f-af1a-29c9a2013996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:49.211365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b151e5-c8f6-407f-af1a-29c9a2013996 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:49.211761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b151e5-c8f6-407f-af1a-29c9a2013996 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:49.212149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b151e5-c8f6-407f-af1a-29c9a2013996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:49.281552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b151e5-c8f6-407f-af1a-29c9a2013996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:49.283581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b151e5-c8f6-407f-af1a-29c9a2013996 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:49.296308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 875/1743] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:33:49.315534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae817a7-ecc0-4561-b287-be09f5bf2496 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:49.317245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-eae817a7-ecc0-4561-b287-be09f5bf2496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:49.389304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae817a7-ecc0-4561-b287-be09f5bf2496 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:49.402116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:49.424905 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b151e5-c8f6-407f-af1a-29c9a2013996 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:49.427978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d2b151e5-c8f6-407f-af1a-29c9a2013996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:49.467861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f8054-3bf5-4fac-83e0-634af9998aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:49.468506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f8054-3bf5-4fac-83e0-634af9998aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:49.470086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f8054-3bf5-4fac-83e0-634af9998aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:49.470199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f8054-3bf5-4fac-83e0-634af9998aec admin admin] PATH_INFO: `/v3/users/20af20cdaa244bd4955338fbf3c05680` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:49.471414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f8054-3bf5-4fac-83e0-634af9998aec admin admin] RBAC: Authorizing `identity:delete_user(user_id=20af20cdaa244bd4955338fbf3c05680)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:49.473606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f8054-3bf5-4fac-83e0-634af9998aec admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:49.529570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b151e5-c8f6-407f-af1a-29c9a2013996 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:49.534548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:49.535181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:49.535457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:49.535724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] PATH_INFO: `/v3/projects/ef4a7a64959d4cc99f25337a45d634be` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:49.536871 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef4a7a64959d4cc99f25337a45d634be)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:49.538877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:49.822381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef4a7a64959d4cc99f25337a45d634be', 'request_id': 'req-9cef3f4d-2a47-4200-b04a-79949483dff0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:49.827831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0d5f8054-3bf5-4fac-83e0-634af9998aec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20af20cdaa244bd4955338fbf3c05680'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:49.830365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0d5f8054-3bf5-4fac-83e0-634af9998aec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20af20cdaa244bd4955338fbf3c05680'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:49.851941 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef4a7a64959d4cc99f25337a45d634be', 'request_id': 'req-9cef3f4d-2a47-4200-b04a-79949483dff0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.003270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0d5f8054-3bf5-4fac-83e0-634af9998aec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20af20cdaa244bd4955338fbf3c05680'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.003270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0d5f8054-3bf5-4fac-83e0-634af9998aec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20af20cdaa244bd4955338fbf3c05680'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.011303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0d5f8054-3bf5-4fac-83e0-634af9998aec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20af20cdaa244bd4955338fbf3c05680'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.030498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 869/1744] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:49 2022] DELETE /identity/v3/users/20af20cdaa244bd4955338fbf3c05680 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:50.037045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1e2a8-1bd9-493f-bf20-eedd4e7f9bb3 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:50.037045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1e2a8-1bd9-493f-bf20-eedd4e7f9bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:50.037045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1e2a8-1bd9-493f-bf20-eedd4e7f9bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:50.056766 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c6d1e2a8-1bd9-493f-bf20-eedd4e7f9bb3 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:50.057949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c6d1e2a8-1bd9-493f-bf20-eedd4e7f9bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:50.115326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef4a7a64959d4cc99f25337a45d634be', 'request_id': 'req-9cef3f4d-2a47-4200-b04a-79949483dff0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.140883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 870/1745] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:50.149104 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1e2a8-1bd9-493f-bf20-eedd4e7f9bb3 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:50.151566 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c6d1e2a8-1bd9-493f-bf20-eedd4e7f9bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:50.159473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] The token cache is being invalidate because project ef4a7a64959d4cc99f25337a45d634be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:50.159768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.169280 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.170609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 876/1746] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:49 2022] DELETE /identity/v3/projects/ef4a7a64959d4cc99f25337a45d634be => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:50.174475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:50.175952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:50.217659 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:50.238826 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1e2a8-1bd9-493f-bf20-eedd4e7f9bb3 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:50.248574 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350f091-7780-4dd1-a44b-66b936fd8397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:50.250246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350f091-7780-4dd1-a44b-66b936fd8397 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:50.250528 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350f091-7780-4dd1-a44b-66b936fd8397 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:50.250803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350f091-7780-4dd1-a44b-66b936fd8397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:50.257256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b350f091-7780-4dd1-a44b-66b936fd8397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:50.270831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:50.295807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350f091-7780-4dd1-a44b-66b936fd8397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:50.299025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350f091-7780-4dd1-a44b-66b936fd8397 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:50.312840 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef3f4d-2a47-4200-b04a-79949483dff0 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:50.320709 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:50.321367 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:50.321644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:50.321913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] PATH_INFO: `/v3/users/81d2440c29904f0484b70f5e648bf05e` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:50.323113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d2440c29904f0484b70f5e648bf05e)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:50.323747 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 871/1747] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:33:50.326596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:50.593732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d2440c29904f0484b70f5e648bf05e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.594655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d2440c29904f0484b70f5e648bf05e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.607977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d2440c29904f0484b70f5e648bf05e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.695959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d2440c29904f0484b70f5e648bf05e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.696595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d2440c29904f0484b70f5e648bf05e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:50.724310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 877/1748] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:50 2022] DELETE /identity/v3/users/81d2440c29904f0484b70f5e648bf05e => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:50.915654 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350f091-7780-4dd1-a44b-66b936fd8397 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:50.917485 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b350f091-7780-4dd1-a44b-66b936fd8397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:50.936726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350f091-7780-4dd1-a44b-66b936fd8397 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:51.069273 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:51.070800 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:51.071563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:51.072873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] PATH_INFO: `/v3/projects/50d06218d5d44aa0826bf22927331bd0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:51.081934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] RBAC: Authorizing `identity:delete_project(project_id=50d06218d5d44aa0826bf22927331bd0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:51.085038 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:51.085038 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:51.085990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:51.103124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e19b-7cc8-4b7e-b2ff-641d65dee665 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:51.111396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:51.112198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:51.112487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:51.112925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] PATH_INFO: `/v3/projects/898020ba548e4fc1b4a83e5b7d047395` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:51.114118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=898020ba548e4fc1b4a83e5b7d047395)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:51.116178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:51.249902 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50d06218d5d44aa0826bf22927331bd0', 'request_id': 'req-7ee5fd50-4099-463c-8a9a-7017fdb522db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:51.257558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50d06218d5d44aa0826bf22927331bd0', 'request_id': 'req-7ee5fd50-4099-463c-8a9a-7017fdb522db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:51.320830 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '898020ba548e4fc1b4a83e5b7d047395', 'request_id': 'req-7e230ee6-fde2-4530-ab0c-5675d7574db3'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:51.328332 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '898020ba548e4fc1b4a83e5b7d047395', 'request_id': 'req-7e230ee6-fde2-4530-ab0c-5675d7574db3'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:51.331025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50d06218d5d44aa0826bf22927331bd0', 'request_id': 'req-7ee5fd50-4099-463c-8a9a-7017fdb522db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:51.342748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] The token cache is being invalidate because project 50d06218d5d44aa0826bf22927331bd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:51.343122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:51.343838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7ee5fd50-4099-463c-8a9a-7017fdb522db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:51.345605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 872/1749] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:50 2022] DELETE /identity/v3/projects/50d06218d5d44aa0826bf22927331bd0 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:51.350258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061cbc4-cecf-4918-b40c-c9d44d7496e6 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:51.350539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061cbc4-cecf-4918-b40c-c9d44d7496e6 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:51.350808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061cbc4-cecf-4918-b40c-c9d44d7496e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:51.373134 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-2061cbc4-cecf-4918-b40c-c9d44d7496e6 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:51.374024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2061cbc4-cecf-4918-b40c-c9d44d7496e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:51.416435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '898020ba548e4fc1b4a83e5b7d047395', 'request_id': 'req-7e230ee6-fde2-4530-ab0c-5675d7574db3'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:51.432023 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] The token cache is being invalidate because project 898020ba548e4fc1b4a83e5b7d047395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:51.432316 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:51.433786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7e230ee6-fde2-4530-ab0c-5675d7574db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:51.439695 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 878/1750] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:51 2022] DELETE /identity/v3/projects/898020ba548e4fc1b4a83e5b7d047395 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:51.441230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f48d5-f63c-4249-8510-4da80cd2f6b8 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:51.441605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f48d5-f63c-4249-8510-4da80cd2f6b8 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:51.441973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f48d5-f63c-4249-8510-4da80cd2f6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:51.466231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-480f48d5-f63c-4249-8510-4da80cd2f6b8 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:51.468840 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-480f48d5-f63c-4249-8510-4da80cd2f6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:51.470387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 873/1751] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:51.479157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99898ea3-f045-42c6-9c2a-8883f6bf98e7 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:51.480127 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99898ea3-f045-42c6-9c2a-8883f6bf98e7 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:51.480446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99898ea3-f045-42c6-9c2a-8883f6bf98e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:51.484055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 879/1752] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:51.492829 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96481c-55f9-43dd-a3f4-b08f6d03b6f6 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:51.493296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96481c-55f9-43dd-a3f4-b08f6d03b6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:51.493476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96481c-55f9-43dd-a3f4-b08f6d03b6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:51.514750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-99898ea3-f045-42c6-9c2a-8883f6bf98e7 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:51.515615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-1b96481c-55f9-43dd-a3f4-b08f6d03b6f6 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:51.515965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-99898ea3-f045-42c6-9c2a-8883f6bf98e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:51.516880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1b96481c-55f9-43dd-a3f4-b08f6d03b6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:51.525144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 874/1753] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:51.529452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 880/1754] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:51.530263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99898ea3-f045-42c6-9c2a-8883f6bf98e7 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:51.530460 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96481c-55f9-43dd-a3f4-b08f6d03b6f6 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:51.536656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:51.540764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96481c-55f9-43dd-a3f4-b08f6d03b6f6 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:51.541201 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99898ea3-f045-42c6-9c2a-8883f6bf98e7 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:51.549698 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738e385-570d-4beb-8d41-693f4a3d2d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:51.549698 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738e385-570d-4beb-8d41-693f4a3d2d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:51.549698 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738e385-570d-4beb-8d41-693f4a3d2d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:51.549698 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738e385-570d-4beb-8d41-693f4a3d2d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:51.550032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738e385-570d-4beb-8d41-693f4a3d2d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:33:51.550032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738e385-570d-4beb-8d41-693f4a3d2d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:51.551919 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738e385-570d-4beb-8d41-693f4a3d2d05 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:51.553906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb2ee9-af5c-4a1a-96da-090d27935e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:51.554570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb2ee9-af5c-4a1a-96da-090d27935e50 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:51.554849 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb2ee9-af5c-4a1a-96da-090d27935e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:51.555122 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb2ee9-af5c-4a1a-96da-090d27935e50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:51.555785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb2ee9-af5c-4a1a-96da-090d27935e50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:33:51.556106 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb2ee9-af5c-4a1a-96da-090d27935e50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:51.557774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb2ee9-af5c-4a1a-96da-090d27935e50 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:51.568221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:51.568413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 875/1755] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:33:51.570185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738e385-570d-4beb-8d41-693f4a3d2d05 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:51.575089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 881/1756] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:33:51.580828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb2ee9-af5c-4a1a-96da-090d27935e50 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:51.593664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738e385-570d-4beb-8d41-693f4a3d2d05 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:51.598746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb2ee9-af5c-4a1a-96da-090d27935e50 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:51.599618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a977ba-0877-4cba-b388-d5fb4e8138a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:51.600600 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a977ba-0877-4cba-b388-d5fb4e8138a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:51.601148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a977ba-0877-4cba-b388-d5fb4e8138a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:51.601660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a977ba-0877-4cba-b388-d5fb4e8138a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:51.602718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a977ba-0877-4cba-b388-d5fb4e8138a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:51.607906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a977ba-0877-4cba-b388-d5fb4e8138a2 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:51.630203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75a28c-73de-46ba-b937-c48fc25349b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:51.630203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a28c-73de-46ba-b937-c48fc25349b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:51.630203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a28c-73de-46ba-b937-c48fc25349b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:51.630203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a28c-73de-46ba-b937-c48fc25349b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:51.632432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75a28c-73de-46ba-b937-c48fc25349b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:51.632432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75a28c-73de-46ba-b937-c48fc25349b6 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:51.881642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 882/1757] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:51 2022] POST /identity/v3/projects => generated 375 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:51.886519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 876/1758] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:51 2022] POST /identity/v3/projects => generated 385 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:51.886667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75a28c-73de-46ba-b937-c48fc25349b6 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:51.896086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a977ba-0877-4cba-b388-d5fb4e8138a2 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:51.972615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75a28c-73de-46ba-b937-c48fc25349b6 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:51.991453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a977ba-0877-4cba-b388-d5fb4e8138a2 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.000431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57c785-21dd-4071-978c-1fa22383827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.001072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57c785-21dd-4071-978c-1fa22383827b admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.001519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57c785-21dd-4071-978c-1fa22383827b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.001807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57c785-21dd-4071-978c-1fa22383827b admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.002774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57c785-21dd-4071-978c-1fa22383827b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.005302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57c785-21dd-4071-978c-1fa22383827b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.079040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2c514-6aa3-4737-9c5e-e75029b99640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.079674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2c514-6aa3-4737-9c5e-e75029b99640 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.079954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2c514-6aa3-4737-9c5e-e75029b99640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.080230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2c514-6aa3-4737-9c5e-e75029b99640 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.080881 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2c514-6aa3-4737-9c5e-e75029b99640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.082895 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2c514-6aa3-4737-9c5e-e75029b99640 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.102288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 877/1759] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:51 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:52.107902 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57c785-21dd-4071-978c-1fa22383827b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:52.121911 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57c785-21dd-4071-978c-1fa22383827b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.128061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df52864-ab6d-4cc2-b36e-1d07c6c7e97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.130331 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df52864-ab6d-4cc2-b36e-1d07c6c7e97f admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.130331 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df52864-ab6d-4cc2-b36e-1d07c6c7e97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.137413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df52864-ab6d-4cc2-b36e-1d07c6c7e97f admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.137413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df52864-ab6d-4cc2-b36e-1d07c6c7e97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.137413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df52864-ab6d-4cc2-b36e-1d07c6c7e97f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.144519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 878/1760] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:52.148288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df52864-ab6d-4cc2-b36e-1d07c6c7e97f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:52.159153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df52864-ab6d-4cc2-b36e-1d07c6c7e97f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.167184 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db2af2-a039-4932-803d-ac3fdca500b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.167822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2af2-a039-4932-803d-ac3fdca500b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.168176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2af2-a039-4932-803d-ac3fdca500b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.171695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db2af2-a039-4932-803d-ac3fdca500b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.171695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db2af2-a039-4932-803d-ac3fdca500b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.171695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db2af2-a039-4932-803d-ac3fdca500b5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.186177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 879/1761] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:33:52.187535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db2af2-a039-4932-803d-ac3fdca500b5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:52.195016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 883/1762] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:51 2022] POST /identity/v3/users => generated 288 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:52.202635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db2af2-a039-4932-803d-ac3fdca500b5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.204760 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2c514-6aa3-4737-9c5e-e75029b99640 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:52.209373 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76ac68-3507-4bdc-a518-e38a465655a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.210062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ac68-3507-4bdc-a518-e38a465655a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.210346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ac68-3507-4bdc-a518-e38a465655a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.210799 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76ac68-3507-4bdc-a518-e38a465655a4 admin admin] PATH_INFO: `/v3/projects/1c457a14369446769d55fbb2797e9efa/users/907d456f9b4e404d8059bcacb6100d53/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.217491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2c514-6aa3-4737-9c5e-e75029b99640 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.229359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76ac68-3507-4bdc-a518-e38a465655a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c457a14369446769d55fbb2797e9efa, user_id=907d456f9b4e404d8059bcacb6100d53, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.229595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73eacf-2351-47e2-b45e-4ddc61d929a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.230380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73eacf-2351-47e2-b45e-4ddc61d929a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.230721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73eacf-2351-47e2-b45e-4ddc61d929a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.235092 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73eacf-2351-47e2-b45e-4ddc61d929a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.235092 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73eacf-2351-47e2-b45e-4ddc61d929a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.235301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76ac68-3507-4bdc-a518-e38a465655a4 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.235392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73eacf-2351-47e2-b45e-4ddc61d929a4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.254298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 884/1763] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:52.255578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73eacf-2351-47e2-b45e-4ddc61d929a4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:52.265891 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73eacf-2351-47e2-b45e-4ddc61d929a4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.270730 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c129-3a78-49c8-b67a-e91199b2263a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.271406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c129-3a78-49c8-b67a-e91199b2263a admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.271774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c129-3a78-49c8-b67a-e91199b2263a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.272114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c129-3a78-49c8-b67a-e91199b2263a admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.272835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4c129-3a78-49c8-b67a-e91199b2263a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.274517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4c129-3a78-49c8-b67a-e91199b2263a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.286094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 885/1764] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:52.289952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c129-3a78-49c8-b67a-e91199b2263a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:52.302368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c129-3a78-49c8-b67a-e91199b2263a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.315845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 880/1765] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:52 2022] PUT /identity/v3/projects/1c457a14369446769d55fbb2797e9efa/users/907d456f9b4e404d8059bcacb6100d53/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:52.319528 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce512f-3ca1-4806-9399-0a89d0218628 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.320407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce512f-3ca1-4806-9399-0a89d0218628 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.320875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce512f-3ca1-4806-9399-0a89d0218628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.383615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-bfce512f-3ca1-4806-9399-0a89d0218628 None None] MFA Rules not processed for user `907d456f9b4e404d8059bcacb6100d53`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:52.384848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-bfce512f-3ca1-4806-9399-0a89d0218628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:52.395678 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa2bac-aed8-4372-bc12-87413978f08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.396472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa2bac-aed8-4372-bc12-87413978f08e admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.397317 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa2bac-aed8-4372-bc12-87413978f08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.397713 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa2bac-aed8-4372-bc12-87413978f08e admin admin] PATH_INFO: `/v3/projects/1f4ceae6c7344ab984deec36950db77d/users/1a1067e91a1548bca2e70462d48b50d5/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.421171 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:52.425001 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa2bac-aed8-4372-bc12-87413978f08e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4ceae6c7344ab984deec36950db77d, user_id=1a1067e91a1548bca2e70462d48b50d5, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.428503 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa2bac-aed8-4372-bc12-87413978f08e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.473138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 881/1766] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:52.477695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce512f-3ca1-4806-9399-0a89d0218628 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:52.484379 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:52.494527 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce512f-3ca1-4806-9399-0a89d0218628 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.500238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00fc4-2dba-47a8-ac50-59ef1e3dd46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.500855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00fc4-2dba-47a8-ac50-59ef1e3dd46f admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.501132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00fc4-2dba-47a8-ac50-59ef1e3dd46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.501406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f00fc4-2dba-47a8-ac50-59ef1e3dd46f admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.502105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f00fc4-2dba-47a8-ac50-59ef1e3dd46f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.504304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f00fc4-2dba-47a8-ac50-59ef1e3dd46f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.576704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 886/1767] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:52 2022] PUT /identity/v3/projects/1f4ceae6c7344ab984deec36950db77d/users/1a1067e91a1548bca2e70462d48b50d5/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:52.589374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069793f8-2e08-4d53-946b-261b5ce7583f None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.589756 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069793f8-2e08-4d53-946b-261b5ce7583f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.590165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069793f8-2e08-4d53-946b-261b5ce7583f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.603553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 882/1768] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:52 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:33:52.611155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00fc4-2dba-47a8-ac50-59ef1e3dd46f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:52.624604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-069793f8-2e08-4d53-946b-261b5ce7583f None None] MFA Rules not processed for user `1a1067e91a1548bca2e70462d48b50d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:52.630133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-069793f8-2e08-4d53-946b-261b5ce7583f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:52.714395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f00fc4-2dba-47a8-ac50-59ef1e3dd46f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.729998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b97897-23a7-4dc6-9ee2-55b8080f9781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.735831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b97897-23a7-4dc6-9ee2-55b8080f9781 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.735831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b97897-23a7-4dc6-9ee2-55b8080f9781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.735831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b97897-23a7-4dc6-9ee2-55b8080f9781 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.735831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b97897-23a7-4dc6-9ee2-55b8080f9781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.739753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b97897-23a7-4dc6-9ee2-55b8080f9781 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.740638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 887/1769] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:52.755137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069793f8-2e08-4d53-946b-261b5ce7583f None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:52.768387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069793f8-2e08-4d53-946b-261b5ce7583f None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.779558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724099a6-a39a-4218-8631-23bf64dc5a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.780352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724099a6-a39a-4218-8631-23bf64dc5a2c admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.780731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724099a6-a39a-4218-8631-23bf64dc5a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.781106 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724099a6-a39a-4218-8631-23bf64dc5a2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.781858 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724099a6-a39a-4218-8631-23bf64dc5a2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.784022 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724099a6-a39a-4218-8631-23bf64dc5a2c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.932220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 883/1770] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:52 2022] POST /identity/v3/users => generated 293 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:33:52.936760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b97897-23a7-4dc6-9ee2-55b8080f9781 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:52.950237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b97897-23a7-4dc6-9ee2-55b8080f9781 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.955419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c31c3-c634-4a80-8f3b-89e06bf7d176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:52.956033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c31c3-c634-4a80-8f3b-89e06bf7d176 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:52.956601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c31c3-c634-4a80-8f3b-89e06bf7d176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:52.956601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c31c3-c634-4a80-8f3b-89e06bf7d176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:52.957327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c31c3-c634-4a80-8f3b-89e06bf7d176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:52.958973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c31c3-c634-4a80-8f3b-89e06bf7d176 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:52.969288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 884/1771] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:52.973083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c31c3-c634-4a80-8f3b-89e06bf7d176 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:52.985219 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c31c3-c634-4a80-8f3b-89e06bf7d176 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:52.996439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 888/1772] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:52 2022] POST /identity/v3/projects => generated 377 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:53.002843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724099a6-a39a-4218-8631-23bf64dc5a2c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:53.011985 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841a9e9-9a91-4d88-b441-6aa744eb1687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:53.012626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841a9e9-9a91-4d88-b441-6aa744eb1687 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:53.012968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841a9e9-9a91-4d88-b441-6aa744eb1687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:53.013221 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841a9e9-9a91-4d88-b441-6aa744eb1687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:53.013916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841a9e9-9a91-4d88-b441-6aa744eb1687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:53.015660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841a9e9-9a91-4d88-b441-6aa744eb1687 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:53.029039 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 885/1773] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:53.045344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841a9e9-9a91-4d88-b441-6aa744eb1687 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:53.088706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841a9e9-9a91-4d88-b441-6aa744eb1687 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:53.128988 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724099a6-a39a-4218-8631-23bf64dc5a2c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:53.148496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479281cd-1ffd-478f-9cec-0eed222a8d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:53.151644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479281cd-1ffd-478f-9cec-0eed222a8d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:53.153508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479281cd-1ffd-478f-9cec-0eed222a8d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:53.155146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479281cd-1ffd-478f-9cec-0eed222a8d2d admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:53.156813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479281cd-1ffd-478f-9cec-0eed222a8d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:53.161714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479281cd-1ffd-478f-9cec-0eed222a8d2d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:53.180691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88212733-ceaf-4dbd-9be9-655168789cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:53.181527 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88212733-ceaf-4dbd-9be9-655168789cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:53.181866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88212733-ceaf-4dbd-9be9-655168789cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:53.182687 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88212733-ceaf-4dbd-9be9-655168789cc9 admin admin] PATH_INFO: `/v3/projects/07588c864f03445297da2a7c31754278/users/55ebea16e1a6440f9b9ce369b9e9f186/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:53.202467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88212733-ceaf-4dbd-9be9-655168789cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07588c864f03445297da2a7c31754278, user_id=55ebea16e1a6440f9b9ce369b9e9f186, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:53.205373 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88212733-ceaf-4dbd-9be9-655168789cc9 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:53.319116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 889/1774] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:53 2022] POST /identity/v3/users => generated 289 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:53.322062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479281cd-1ffd-478f-9cec-0eed222a8d2d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:53.336264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479281cd-1ffd-478f-9cec-0eed222a8d2d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:53.341660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86ff7b-18d9-4681-86f4-ea274a09c618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:53.342311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86ff7b-18d9-4681-86f4-ea274a09c618 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:53.342591 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86ff7b-18d9-4681-86f4-ea274a09c618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:53.342864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86ff7b-18d9-4681-86f4-ea274a09c618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:53.343715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86ff7b-18d9-4681-86f4-ea274a09c618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:53.345166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d86ff7b-18d9-4681-86f4-ea274a09c618 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:53.359594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 890/1775] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:53.364403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86ff7b-18d9-4681-86f4-ea274a09c618 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:53.369261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 886/1776] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:53 2022] PUT /identity/v3/projects/07588c864f03445297da2a7c31754278/users/55ebea16e1a6440f9b9ce369b9e9f186/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:53.379730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbcae6-8bc2-4c91-8009-d5186a8c0757 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:53.380029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbcae6-8bc2-4c91-8009-d5186a8c0757 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:53.384850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbcae6-8bc2-4c91-8009-d5186a8c0757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:53.385594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d86ff7b-18d9-4681-86f4-ea274a09c618 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:53.411158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:53.457817 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-15dbcae6-8bc2-4c91-8009-d5186a8c0757 None None] MFA Rules not processed for user `55ebea16e1a6440f9b9ce369b9e9f186`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:53.459227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-15dbcae6-8bc2-4c91-8009-d5186a8c0757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:53.487228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:53.498720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b51f6-ac3e-423e-80c2-507928cb8dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:53.499548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b51f6-ac3e-423e-80c2-507928cb8dcd admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:53.500064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b51f6-ac3e-423e-80c2-507928cb8dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:53.500478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b51f6-ac3e-423e-80c2-507928cb8dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:53.501503 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b51f6-ac3e-423e-80c2-507928cb8dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:53.504034 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b51f6-ac3e-423e-80c2-507928cb8dcd admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:53.520960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 891/1777] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:53.524959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b51f6-ac3e-423e-80c2-507928cb8dcd admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:53.555769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b51f6-ac3e-423e-80c2-507928cb8dcd admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:53.564761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7d74a-0a9c-4fa9-b095-8443dbcbf27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:53.565706 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7d74a-0a9c-4fa9-b095-8443dbcbf27d admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:53.566181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7d74a-0a9c-4fa9-b095-8443dbcbf27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:53.566814 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7d74a-0a9c-4fa9-b095-8443dbcbf27d admin admin] PATH_INFO: `/v3/projects/43b37b5d497f453c9dbe8a94a926c312/users/1e986543b1fc4218b0182d5f4204d72a/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:53.588152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 887/1778] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:53.595504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbcae6-8bc2-4c91-8009-d5186a8c0757 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:53.595919 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7d74a-0a9c-4fa9-b095-8443dbcbf27d admin admin] RBAC: Authorizing `identity:create_grant(project_id=43b37b5d497f453c9dbe8a94a926c312, user_id=1e986543b1fc4218b0182d5f4204d72a, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:53.603305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7d74a-0a9c-4fa9-b095-8443dbcbf27d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:53.624379 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbcae6-8bc2-4c91-8009-d5186a8c0757 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:53.630446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927453b4-bc69-4420-8a5d-7ee17cf8f267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:53.631060 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927453b4-bc69-4420-8a5d-7ee17cf8f267 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:53.634865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927453b4-bc69-4420-8a5d-7ee17cf8f267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:53.634865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927453b4-bc69-4420-8a5d-7ee17cf8f267 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:53.634865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927453b4-bc69-4420-8a5d-7ee17cf8f267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:53.634865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927453b4-bc69-4420-8a5d-7ee17cf8f267 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:53.701283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 892/1779] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:53 2022] PUT /identity/v3/projects/43b37b5d497f453c9dbe8a94a926c312/users/1e986543b1fc4218b0182d5f4204d72a/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:53.703112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7d74a-0a9c-4fa9-b095-8443dbcbf27d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:53.707076 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2fb7d74a-0a9c-4fa9-b095-8443dbcbf27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:53.772751 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7d74a-0a9c-4fa9-b095-8443dbcbf27d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:53.777730 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:53.778987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:53.779264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:53.779533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] PATH_INFO: `/v3/users/f33275c5521842349cc00e470e60b822` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:53.780659 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f33275c5521842349cc00e470e60b822)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:53.782623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:53.806174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 888/1780] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:53 2022] POST /identity/v3/projects => generated 383 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:53.807450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1956ee-2cc5-4742-8e3f-035cf27411a8 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:53.807727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1956ee-2cc5-4742-8e3f-035cf27411a8 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:53.808103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1956ee-2cc5-4742-8e3f-035cf27411a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:53.835855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-ff1956ee-2cc5-4742-8e3f-035cf27411a8 None None] MFA Rules not processed for user `1e986543b1fc4218b0182d5f4204d72a`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:53.837116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ff1956ee-2cc5-4742-8e3f-035cf27411a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:53.903203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 889/1781] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:53.903203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1956ee-2cc5-4742-8e3f-035cf27411a8 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:53.970085 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1956ee-2cc5-4742-8e3f-035cf27411a8 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:53.976091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c3b7a-d67f-4051-850a-167632a9c70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:53.976913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c3b7a-d67f-4051-850a-167632a9c70d admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:53.977286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c3b7a-d67f-4051-850a-167632a9c70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:53.977704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c3b7a-d67f-4051-850a-167632a9c70d admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:53.978524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c3b7a-d67f-4051-850a-167632a9c70d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:53.981268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c3b7a-d67f-4051-850a-167632a9c70d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:54.031682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f33275c5521842349cc00e470e60b822'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:54.032450 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f33275c5521842349cc00e470e60b822'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:54.039727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f33275c5521842349cc00e470e60b822'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:54.096437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 890/1782] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:53 2022] POST /identity/v3/users => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:54.098016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c3b7a-d67f-4051-850a-167632a9c70d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:54.100255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-109c3b7a-d67f-4051-850a-167632a9c70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:54.114472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c3b7a-d67f-4051-850a-167632a9c70d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:54.119034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73c111-41bd-4f2c-8ae8-0de3899fd001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:54.119751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73c111-41bd-4f2c-8ae8-0de3899fd001 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.120034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73c111-41bd-4f2c-8ae8-0de3899fd001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.120304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73c111-41bd-4f2c-8ae8-0de3899fd001 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.120933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73c111-41bd-4f2c-8ae8-0de3899fd001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:54.123581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73c111-41bd-4f2c-8ae8-0de3899fd001 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:54.124820 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f33275c5521842349cc00e470e60b822'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:54.125443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f33275c5521842349cc00e470e60b822'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:54.144275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 893/1783] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:33:53 2022] DELETE /identity/v3/users/f33275c5521842349cc00e470e60b822 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:54.145497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:54.146980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:54.193846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:54.212187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6127ad-3e1e-4e0f-81a0-f6fddaff1378 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:54.218054 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d16eb-4803-4d60-8173-f34f652fa677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:54.218791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d16eb-4803-4d60-8173-f34f652fa677 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.219084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d16eb-4803-4d60-8173-f34f652fa677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.219360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d16eb-4803-4d60-8173-f34f652fa677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.222481 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d16eb-4803-4d60-8173-f34f652fa677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:54.226061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d16eb-4803-4d60-8173-f34f652fa677 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:54.231449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 891/1784] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:54 2022] POST /identity/v3/projects => generated 375 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:54.235841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73c111-41bd-4f2c-8ae8-0de3899fd001 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:54.266066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 894/1785] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:54.271012 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d16eb-4803-4d60-8173-f34f652fa677 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:54.280283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:54.284195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d16eb-4803-4d60-8173-f34f652fa677 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:54.318404 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73c111-41bd-4f2c-8ae8-0de3899fd001 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:54.324550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86277381-a72f-45f6-9cae-a753331c6d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:54.325877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86277381-a72f-45f6-9cae-a753331c6d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.326280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86277381-a72f-45f6-9cae-a753331c6d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.326647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86277381-a72f-45f6-9cae-a753331c6d8f admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.327392 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86277381-a72f-45f6-9cae-a753331c6d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:54.329452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86277381-a72f-45f6-9cae-a753331c6d8f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:54.349467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9b7fc-63ea-4018-bd66-06aecf9cdb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:54.350278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9b7fc-63ea-4018-bd66-06aecf9cdb9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.350600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9b7fc-63ea-4018-bd66-06aecf9cdb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.350882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9b7fc-63ea-4018-bd66-06aecf9cdb9f admin admin] PATH_INFO: `/v3/projects/678f1c0487034471ba012ac79b80a37f/users/23d95a7e822146a683cf7a9156e01e16/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.365848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9b7fc-63ea-4018-bd66-06aecf9cdb9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=678f1c0487034471ba012ac79b80a37f, user_id=23d95a7e822146a683cf7a9156e01e16, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:54.368579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9b7fc-63ea-4018-bd66-06aecf9cdb9f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:54.449724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 892/1786] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:54 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:54.453671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86277381-a72f-45f6-9cae-a753331c6d8f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:54.463179 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86277381-a72f-45f6-9cae-a753331c6d8f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:54.467916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd617e16-594e-4aa4-a5bf-9dba8767f78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:54.468521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd617e16-594e-4aa4-a5bf-9dba8767f78d admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.468804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd617e16-594e-4aa4-a5bf-9dba8767f78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.469079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd617e16-594e-4aa4-a5bf-9dba8767f78d admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.469719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd617e16-594e-4aa4-a5bf-9dba8767f78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:54.471360 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd617e16-594e-4aa4-a5bf-9dba8767f78d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:54.479725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 895/1787] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:54 2022] PUT /identity/v3/projects/678f1c0487034471ba012ac79b80a37f/users/23d95a7e822146a683cf7a9156e01e16/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:54.483215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9b7fc-63ea-4018-bd66-06aecf9cdb9f admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:54.484437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-fbf9b7fc-63ea-4018-bd66-06aecf9cdb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:54.485513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 893/1788] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:33:54.486460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd617e16-594e-4aa4-a5bf-9dba8767f78d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:54.498849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd617e16-594e-4aa4-a5bf-9dba8767f78d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:54.553024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9b7fc-63ea-4018-bd66-06aecf9cdb9f admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:54.557793 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:54.558455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.558733 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.558998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] PATH_INFO: `/v3/projects/ce28abbc9ca441e5828551c3ee01d79c` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.560185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce28abbc9ca441e5828551c3ee01d79c)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:54.561501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785839d-f11c-40bf-99b9-7c238e3c619f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:54.562088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:54.562213 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785839d-f11c-40bf-99b9-7c238e3c619f admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.562401 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785839d-f11c-40bf-99b9-7c238e3c619f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.562675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785839d-f11c-40bf-99b9-7c238e3c619f admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.563448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785839d-f11c-40bf-99b9-7c238e3c619f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:54.564927 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785839d-f11c-40bf-99b9-7c238e3c619f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:54.576843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 894/1789] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:54.581062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785839d-f11c-40bf-99b9-7c238e3c619f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:54.591684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785839d-f11c-40bf-99b9-7c238e3c619f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:54.596599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b53bef-0f9e-4c7b-bdd3-7aae25b0509c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:54.597266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b53bef-0f9e-4c7b-bdd3-7aae25b0509c admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.597544 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b53bef-0f9e-4c7b-bdd3-7aae25b0509c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.597818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b53bef-0f9e-4c7b-bdd3-7aae25b0509c admin admin] PATH_INFO: `/v3/projects/423f9941f3e44a779c73c8622d924c16/users/e7027ef062814ddf8818fcb02937499e/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.613337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b53bef-0f9e-4c7b-bdd3-7aae25b0509c admin admin] RBAC: Authorizing `identity:create_grant(project_id=423f9941f3e44a779c73c8622d924c16, user_id=e7027ef062814ddf8818fcb02937499e, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:54.616082 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b53bef-0f9e-4c7b-bdd3-7aae25b0509c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:54.709640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 895/1790] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:54 2022] PUT /identity/v3/projects/423f9941f3e44a779c73c8622d924c16/users/e7027ef062814ddf8818fcb02937499e/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:54.709640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b53bef-0f9e-4c7b-bdd3-7aae25b0509c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:54.712051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce28abbc9ca441e5828551c3ee01d79c', 'request_id': 'req-52282ac1-aa1a-4a17-a32a-ed25f82606ce'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:54.719736 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce28abbc9ca441e5828551c3ee01d79c', 'request_id': 'req-52282ac1-aa1a-4a17-a32a-ed25f82606ce'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:54.782062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b53bef-0f9e-4c7b-bdd3-7aae25b0509c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:54.787361 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a925a45-c4aa-437f-8b52-bf1c3e07f6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:54.787770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a925a45-c4aa-437f-8b52-bf1c3e07f6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.788045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a925a45-c4aa-437f-8b52-bf1c3e07f6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.788879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a925a45-c4aa-437f-8b52-bf1c3e07f6ad admin admin] PATH_INFO: `/v3/projects/678f1c0487034471ba012ac79b80a37f/users/23d95a7e822146a683cf7a9156e01e16/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.790616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a925a45-c4aa-437f-8b52-bf1c3e07f6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=678f1c0487034471ba012ac79b80a37f, user_id=23d95a7e822146a683cf7a9156e01e16, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:54.795103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a925a45-c4aa-437f-8b52-bf1c3e07f6ad admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:54.815413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 896/1791] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:54 2022] PUT /identity/v3/projects/678f1c0487034471ba012ac79b80a37f/users/23d95a7e822146a683cf7a9156e01e16/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:54.816318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a925a45-c4aa-437f-8b52-bf1c3e07f6ad admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:54.833541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce28abbc9ca441e5828551c3ee01d79c', 'request_id': 'req-52282ac1-aa1a-4a17-a32a-ed25f82606ce'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:54.849124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] The token cache is being invalidate because project ce28abbc9ca441e5828551c3ee01d79c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:33:54.849440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:54.850404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-52282ac1-aa1a-4a17-a32a-ed25f82606ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:33:54.855577 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 896/1792] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:33:54 2022] DELETE /identity/v3/projects/ce28abbc9ca441e5828551c3ee01d79c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:33:54.856980 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f12cc-afe2-4ace-9ec2-e931dcfabc5f None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.857269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f12cc-afe2-4ace-9ec2-e931dcfabc5f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.857542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f12cc-afe2-4ace-9ec2-e931dcfabc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.896152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9b4f12cc-afe2-4ace-9ec2-e931dcfabc5f None None] MFA Rules not processed for user `23d95a7e822146a683cf7a9156e01e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:54.897804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9b4f12cc-afe2-4ace-9ec2-e931dcfabc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:54.909065 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a925a45-c4aa-437f-8b52-bf1c3e07f6ad admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:54.914305 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a3d42-9d50-4344-b9bf-1d79f1334507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:54.915181 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a3d42-9d50-4344-b9bf-1d79f1334507 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.915815 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a3d42-9d50-4344-b9bf-1d79f1334507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.916303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a3d42-9d50-4344-b9bf-1d79f1334507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.917196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a3d42-9d50-4344-b9bf-1d79f1334507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:54.919095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a3d42-9d50-4344-b9bf-1d79f1334507 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:54.940537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 897/1793] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:54.942162 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9c542-8b76-4ba7-8d46-c9b7f71b3cd2 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.942600 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9c542-8b76-4ba7-8d46-c9b7f71b3cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.942880 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9c542-8b76-4ba7-8d46-c9b7f71b3cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.959347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:54.966573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-43b9c542-8b76-4ba7-8d46-c9b7f71b3cd2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:54.968030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-43b9c542-8b76-4ba7-8d46-c9b7f71b3cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:54.990549 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 898/1795] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:54.990739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 897/1795] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:54.990842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9c542-8b76-4ba7-8d46-c9b7f71b3cd2 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:54.990842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-43b9c542-8b76-4ba7-8d46-c9b7f71b3cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:54.992145 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977979a-ac91-4c82-b81b-f8525bdc8476 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:54.992437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977979a-ac91-4c82-b81b-f8525bdc8476 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:54.992716 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977979a-ac91-4c82-b81b-f8525bdc8476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:54.999323 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9c542-8b76-4ba7-8d46-c9b7f71b3cd2 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:55.004505 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9912d09-0a37-40f4-af0d-433139670bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:55.005161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9912d09-0a37-40f4-af0d-433139670bec admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.005435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9912d09-0a37-40f4-af0d-433139670bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.005725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9912d09-0a37-40f4-af0d-433139670bec admin admin] PATH_INFO: `/v3/projects/423f9941f3e44a779c73c8622d924c16/users/e7027ef062814ddf8818fcb02937499e/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.007694 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9912d09-0a37-40f4-af0d-433139670bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=423f9941f3e44a779c73c8622d924c16, user_id=e7027ef062814ddf8818fcb02937499e, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:55.010725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9912d09-0a37-40f4-af0d-433139670bec admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:55.015262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-f977979a-ac91-4c82-b81b-f8525bdc8476 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:55.016479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f977979a-ac91-4c82-b81b-f8525bdc8476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:55.023407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 899/1796] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:54 2022] PUT /identity/v3/projects/423f9941f3e44a779c73c8622d924c16/users/e7027ef062814ddf8818fcb02937499e/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:55.024288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9912d09-0a37-40f4-af0d-433139670bec admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:55.025902 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d9912d09-0a37-40f4-af0d-433139670bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:55.043598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:55.096371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9912d09-0a37-40f4-af0d-433139670bec admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:55.097598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 898/1797] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:33:55.099168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca25750-be68-49c0-9f6b-de08562034a4 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.099619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca25750-be68-49c0-9f6b-de08562034a4 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.099893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca25750-be68-49c0-9f6b-de08562034a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.117167 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a20ab-f4ff-42b6-acd1-c91aaabd6dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:55.117954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a20ab-f4ff-42b6-acd1-c91aaabd6dbe service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.118383 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a20ab-f4ff-42b6-acd1-c91aaabd6dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.118756 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a20ab-f4ff-42b6-acd1-c91aaabd6dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.120807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d42a20ab-f4ff-42b6-acd1-c91aaabd6dbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:55.131515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7ca25750-be68-49c0-9f6b-de08562034a4 None None] MFA Rules not processed for user `e7027ef062814ddf8818fcb02937499e`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:55.132791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7ca25750-be68-49c0-9f6b-de08562034a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:55.189893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a20ab-f4ff-42b6-acd1-c91aaabd6dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:55.191458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a20ab-f4ff-42b6-acd1-c91aaabd6dbe service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:55.208337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 900/1798] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:55.209530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a20ab-f4ff-42b6-acd1-c91aaabd6dbe service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:55.218691 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 899/1799] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:55.221967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a20ab-f4ff-42b6-acd1-c91aaabd6dbe service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:55.224519 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9aac4-c4ce-47b6-81be-704c23627d5e None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.224846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9aac4-c4ce-47b6-81be-704c23627d5e None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.225131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9aac4-c4ce-47b6-81be-704c23627d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.234482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02a453-5ac6-49ce-8ea6-abc47c27bc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:55.235200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02a453-5ac6-49ce-8ea6-abc47c27bc9e admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.235565 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02a453-5ac6-49ce-8ea6-abc47c27bc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.236044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02a453-5ac6-49ce-8ea6-abc47c27bc9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.236699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02a453-5ac6-49ce-8ea6-abc47c27bc9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:33:55.237001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02a453-5ac6-49ce-8ea6-abc47c27bc9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:55.238672 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02a453-5ac6-49ce-8ea6-abc47c27bc9e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:55.246456 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:55.246588 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-48e9aac4-c4ce-47b6-81be-704c23627d5e None None] MFA Rules not processed for user `22919b0a4b1e4e09b9796a93539547b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:55.247741 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-48e9aac4-c4ce-47b6-81be-704c23627d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:55.252515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 901/1800] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:33:55.253907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02a453-5ac6-49ce-8ea6-abc47c27bc9e admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:55.256147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cd02a453-5ac6-49ce-8ea6-abc47c27bc9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:55.268954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02a453-5ac6-49ce-8ea6-abc47c27bc9e admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:55.274065 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8bed4-0243-4482-ac5e-ab9c2a335ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:55.274667 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8bed4-0243-4482-ac5e-ab9c2a335ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.274939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8bed4-0243-4482-ac5e-ab9c2a335ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.275204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8bed4-0243-4482-ac5e-ab9c2a335ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.276958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-74f8bed4-0243-4482-ac5e-ab9c2a335ee9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:55.319247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 900/1801] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:55.320087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9aac4-c4ce-47b6-81be-704c23627d5e None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:55.336389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9aac4-c4ce-47b6-81be-704c23627d5e None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:55.345231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f28bf-e303-4a1e-8532-fc0339f2ccb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:55.346273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f28bf-e303-4a1e-8532-fc0339f2ccb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.346750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f28bf-e303-4a1e-8532-fc0339f2ccb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.347164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f28bf-e303-4a1e-8532-fc0339f2ccb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.351463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f28bf-e303-4a1e-8532-fc0339f2ccb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:55.351463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08f28bf-e303-4a1e-8532-fc0339f2ccb5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:55.371107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:55.382932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8bed4-0243-4482-ac5e-ab9c2a335ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:55.384815 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8bed4-0243-4482-ac5e-ab9c2a335ee9 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:55.396248 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 902/1802] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:55.535433 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 901/1803] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:33:55 2022] POST /identity/v3/projects => generated 367 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:33:55.535910 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8bed4-0243-4482-ac5e-ab9c2a335ee9 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:55.547598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8bed4-0243-4482-ac5e-ab9c2a335ee9 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:55.610270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bf369-8312-4f23-accc-641b0dbc68ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:55.610886 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bf369-8312-4f23-accc-641b0dbc68ca admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.611185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bf369-8312-4f23-accc-641b0dbc68ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.611453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bf369-8312-4f23-accc-641b0dbc68ca admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.612091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bf369-8312-4f23-accc-641b0dbc68ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:55.614192 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573bf369-8312-4f23-accc-641b0dbc68ca admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:55.706022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 903/1804] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:33:55 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:33:55.707260 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f28bf-e303-4a1e-8532-fc0339f2ccb5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:55.717826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f28bf-e303-4a1e-8532-fc0339f2ccb5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:55.728679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc670d4-c965-4baf-8eda-7d96d48fef4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:55.728679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc670d4-c965-4baf-8eda-7d96d48fef4b admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.728679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc670d4-c965-4baf-8eda-7d96d48fef4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.728679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc670d4-c965-4baf-8eda-7d96d48fef4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.729113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc670d4-c965-4baf-8eda-7d96d48fef4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:55.729113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc670d4-c965-4baf-8eda-7d96d48fef4b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:55.740096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 902/1805] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:55.741399 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bf369-8312-4f23-accc-641b0dbc68ca admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:55.756212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bf369-8312-4f23-accc-641b0dbc68ca admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:55.761127 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c5a276-5fa5-4bf0-9428-53f18b500cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:55.761743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5a276-5fa5-4bf0-9428-53f18b500cfa admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.762066 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5a276-5fa5-4bf0-9428-53f18b500cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.762347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5a276-5fa5-4bf0-9428-53f18b500cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.763575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c5a276-5fa5-4bf0-9428-53f18b500cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:55.765212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c5a276-5fa5-4bf0-9428-53f18b500cfa admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:55.775848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 904/1806] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:33:55.780664 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc670d4-c965-4baf-8eda-7d96d48fef4b admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:55.793350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc670d4-c965-4baf-8eda-7d96d48fef4b admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:55.799073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50385b-41e8-4570-9645-f2ad5be5a934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:55.799741 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50385b-41e8-4570-9645-f2ad5be5a934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.800024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50385b-41e8-4570-9645-f2ad5be5a934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.800306 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50385b-41e8-4570-9645-f2ad5be5a934 admin admin] PATH_INFO: `/v3/projects/9d1af35950b44c53873fc2e47c4ae122/users/378215912ecb4715adc67780c1faf9b6/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.824999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50385b-41e8-4570-9645-f2ad5be5a934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d1af35950b44c53873fc2e47c4ae122, user_id=378215912ecb4715adc67780c1faf9b6, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:55.828365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50385b-41e8-4570-9645-f2ad5be5a934 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:55.897116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 903/1807] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:33:55 2022] PUT /identity/v3/projects/9d1af35950b44c53873fc2e47c4ae122/users/378215912ecb4715adc67780c1faf9b6/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:33:55.907537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87421-9fd1-426b-8d74-148208916341 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:55.907818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87421-9fd1-426b-8d74-148208916341 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:55.908122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87421-9fd1-426b-8d74-148208916341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:55.940835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-7cf87421-9fd1-426b-8d74-148208916341 None None] MFA Rules not processed for user `378215912ecb4715adc67780c1faf9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:33:55.942403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7cf87421-9fd1-426b-8d74-148208916341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:56.067372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 905/1808] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:33:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:33:56.109551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50385b-41e8-4570-9645-f2ad5be5a934 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:56.166537 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50385b-41e8-4570-9645-f2ad5be5a934 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:56.170615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa91ef5-ca52-4875-ad26-5386920bade6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:56.171215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa91ef5-ca52-4875-ad26-5386920bade6 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:56.171492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa91ef5-ca52-4875-ad26-5386920bade6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:56.171764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa91ef5-ca52-4875-ad26-5386920bade6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:56.185753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa91ef5-ca52-4875-ad26-5386920bade6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:56.187522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa91ef5-ca52-4875-ad26-5386920bade6 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:56.198107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 904/1809] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:57.432284 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf87421-9fd1-426b-8d74-148208916341 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:57.438931 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:57.443084 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf87421-9fd1-426b-8d74-148208916341 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:57.446733 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c04f7-248a-43b9-a3ae-08c0f4327e04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:57.447337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c04f7-248a-43b9-a3ae-08c0f4327e04 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:57.447645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c04f7-248a-43b9-a3ae-08c0f4327e04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:57.447913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c04f7-248a-43b9-a3ae-08c0f4327e04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:57.449682 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-178c04f7-248a-43b9-a3ae-08c0f4327e04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:57.519164 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c04f7-248a-43b9-a3ae-08c0f4327e04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:57.520738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c04f7-248a-43b9-a3ae-08c0f4327e04 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:57.530848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 906/1810] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:57.592322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa91ef5-ca52-4875-ad26-5386920bade6 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:57.601783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa91ef5-ca52-4875-ad26-5386920bade6 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:57.606405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82387dd-a524-45c6-ab73-62eea606e162 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:57.607184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82387dd-a524-45c6-ab73-62eea606e162 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:57.607681 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82387dd-a524-45c6-ab73-62eea606e162 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:57.607983 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82387dd-a524-45c6-ab73-62eea606e162 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:57.609946 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a82387dd-a524-45c6-ab73-62eea606e162 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:33:57.631252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:33:57.670232 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82387dd-a524-45c6-ab73-62eea606e162 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:57.671632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82387dd-a524-45c6-ab73-62eea606e162 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:57.680451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 905/1811] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:33:59.652176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c04f7-248a-43b9-a3ae-08c0f4327e04 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:33:59.661576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c04f7-248a-43b9-a3ae-08c0f4327e04 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:33:59.671933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79be546-3dd2-4e68-92a2-025bd78bd2ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:33:59.673033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79be546-3dd2-4e68-92a2-025bd78bd2ae service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:33:59.673522 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79be546-3dd2-4e68-92a2-025bd78bd2ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:33:59.674717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79be546-3dd2-4e68-92a2-025bd78bd2ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:33:59.754246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79be546-3dd2-4e68-92a2-025bd78bd2ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:33:59.756506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79be546-3dd2-4e68-92a2-025bd78bd2ae service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:33:59.769982 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 907/1812] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:33:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:00.072503 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82387dd-a524-45c6-ab73-62eea606e162 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:00.081597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82387dd-a524-45c6-ab73-62eea606e162 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:00.086646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbf7c1-e8b4-4546-b4bb-e44f6a0d180d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:00.087246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbf7c1-e8b4-4546-b4bb-e44f6a0d180d service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:00.087522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbf7c1-e8b4-4546-b4bb-e44f6a0d180d service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:00.087791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbf7c1-e8b4-4546-b4bb-e44f6a0d180d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:00.170731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbf7c1-e8b4-4546-b4bb-e44f6a0d180d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:00.172152 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbf7c1-e8b4-4546-b4bb-e44f6a0d180d service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:00.182071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 906/1813] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:02.273550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79be546-3dd2-4e68-92a2-025bd78bd2ae service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:02.275534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d79be546-3dd2-4e68-92a2-025bd78bd2ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:02.347195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79be546-3dd2-4e68-92a2-025bd78bd2ae service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:02.351780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:02.352431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:02.352707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:02.356682 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] PATH_INFO: `/v3/users/2b17a1ed8c0749189cede37691ba07a2` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:02.356682 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b17a1ed8c0749189cede37691ba07a2)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:02.356682 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:02.623993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:02.630188 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b17a1ed8c0749189cede37691ba07a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:02.630880 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b17a1ed8c0749189cede37691ba07a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:02.764798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b17a1ed8c0749189cede37691ba07a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:02.765479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b17a1ed8c0749189cede37691ba07a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:02.772898 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b17a1ed8c0749189cede37691ba07a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:02.792496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 908/1814] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:02 2022] DELETE /identity/v3/users/2b17a1ed8c0749189cede37691ba07a2 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:02.797183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437fcd9-52c2-4411-91f9-b056c35b0b6c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:02.797606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437fcd9-52c2-4411-91f9-b056c35b0b6c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:02.797751 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d437fcd9-52c2-4411-91f9-b056c35b0b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:02.817004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-d437fcd9-52c2-4411-91f9-b056c35b0b6c None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:02.818429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d437fcd9-52c2-4411-91f9-b056c35b0b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:02.881373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 907/1815] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:02.891422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:02.893380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:02.949043 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d7d23a-c17e-4d93-9b62-bb2231b7a3cb admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:02.953171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d02fd0-db74-46f5-aca4-c8417d7e199d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:02.953773 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d02fd0-db74-46f5-aca4-c8417d7e199d service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:02.954060 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d02fd0-db74-46f5-aca4-c8417d7e199d service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:02.954351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d02fd0-db74-46f5-aca4-c8417d7e199d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:02.963725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d02fd0-db74-46f5-aca4-c8417d7e199d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:02.965160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d02fd0-db74-46f5-aca4-c8417d7e199d service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:02.975842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 909/1816] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:03.588948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437fcd9-52c2-4411-91f9-b056c35b0b6c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:03.590685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d437fcd9-52c2-4411-91f9-b056c35b0b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:03.621314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d437fcd9-52c2-4411-91f9-b056c35b0b6c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:03.627650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:03.628292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:03.628604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:03.628890 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] PATH_INFO: `/v3/projects/f6a37ae65eb34b5cb31549a8864fe86d` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:03.630147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a37ae65eb34b5cb31549a8864fe86d)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:03.632296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:03.775623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a37ae65eb34b5cb31549a8864fe86d', 'request_id': 'req-758f306b-22a1-40f2-89e2-10bac32b24d7'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:03.778352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:03.789146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a37ae65eb34b5cb31549a8864fe86d', 'request_id': 'req-758f306b-22a1-40f2-89e2-10bac32b24d7'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:04.118221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a37ae65eb34b5cb31549a8864fe86d', 'request_id': 'req-758f306b-22a1-40f2-89e2-10bac32b24d7'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:04.136250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] The token cache is being invalidate because project f6a37ae65eb34b5cb31549a8864fe86d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:04.136250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:04.136250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-758f306b-22a1-40f2-89e2-10bac32b24d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:04.136250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 908/1817] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:03 2022] DELETE /identity/v3/projects/f6a37ae65eb34b5cb31549a8864fe86d => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:04.140587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba2a6a-d7a8-4623-9465-3e9c06a432da None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:04.141088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba2a6a-d7a8-4623-9465-3e9c06a432da None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:04.141173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba2a6a-d7a8-4623-9465-3e9c06a432da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:04.161589 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-53ba2a6a-d7a8-4623-9465-3e9c06a432da None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:04.162811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-53ba2a6a-d7a8-4623-9465-3e9c06a432da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:04.241737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 910/1818] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:04.242251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd89c-c38f-4545-9f2d-8aab68c2ad63 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:04.242539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd89c-c38f-4545-9f2d-8aab68c2ad63 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:04.242813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd89c-c38f-4545-9f2d-8aab68c2ad63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:04.267987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9b7bd89c-c38f-4545-9f2d-8aab68c2ad63 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:04.267987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9b7bd89c-c38f-4545-9f2d-8aab68c2ad63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:04.276229 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 909/1819] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:04.282399 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba2a6a-d7a8-4623-9465-3e9c06a432da None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:04.305416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba2a6a-d7a8-4623-9465-3e9c06a432da None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:04.310396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ef2e1-9da6-4a7a-9a46-ed4d24440e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:04.311205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ef2e1-9da6-4a7a-9a46-ed4d24440e23 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:04.311575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ef2e1-9da6-4a7a-9a46-ed4d24440e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:04.311887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ef2e1-9da6-4a7a-9a46-ed4d24440e23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:04.312663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ef2e1-9da6-4a7a-9a46-ed4d24440e23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:34:04.313031 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ef2e1-9da6-4a7a-9a46-ed4d24440e23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:04.315098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ef2e1-9da6-4a7a-9a46-ed4d24440e23 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:04.331452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 911/1820] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:34:04.336011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bd89c-c38f-4545-9f2d-8aab68c2ad63 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:04.346639 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bd89c-c38f-4545-9f2d-8aab68c2ad63 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:04.353000 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca0c3a-2282-4c0a-8209-1a622adc9713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:04.353810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca0c3a-2282-4c0a-8209-1a622adc9713 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:04.354198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca0c3a-2282-4c0a-8209-1a622adc9713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:04.354541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca0c3a-2282-4c0a-8209-1a622adc9713 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:04.355297 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca0c3a-2282-4c0a-8209-1a622adc9713 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:04.357595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca0c3a-2282-4c0a-8209-1a622adc9713 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:04.488813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 910/1821] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:04 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:04.492411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ef2e1-9da6-4a7a-9a46-ed4d24440e23 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:04.504506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ef2e1-9da6-4a7a-9a46-ed4d24440e23 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:04.577496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:04.607110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7d303-89e3-4fc8-be5a-605b68f06c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:04.607751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7d303-89e3-4fc8-be5a-605b68f06c49 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:04.608088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7d303-89e3-4fc8-be5a-605b68f06c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:04.610823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7d303-89e3-4fc8-be5a-605b68f06c49 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:04.612251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb7d303-89e3-4fc8-be5a-605b68f06c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:04.619907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb7d303-89e3-4fc8-be5a-605b68f06c49 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:04.774358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca0c3a-2282-4c0a-8209-1a622adc9713 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:04.777361 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-afca0c3a-2282-4c0a-8209-1a622adc9713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:04.786087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 912/1822] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:04 2022] POST /identity/v3/users => generated 287 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:04.788016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7d303-89e3-4fc8-be5a-605b68f06c49 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:04.789502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca0c3a-2282-4c0a-8209-1a622adc9713 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:04.794036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:04.794995 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:04.794995 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:04.795214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] PATH_INFO: `/v3/users/907d456f9b4e404d8059bcacb6100d53` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:04.796339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=907d456f9b4e404d8059bcacb6100d53)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:04.798556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:04.801094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7d303-89e3-4fc8-be5a-605b68f06c49 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:04.809493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f6bf8-275f-4181-b40f-867adf032c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:04.810207 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f6bf8-275f-4181-b40f-867adf032c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:04.810495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f6bf8-275f-4181-b40f-867adf032c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:04.810832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f6bf8-275f-4181-b40f-867adf032c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:04.811674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f6bf8-275f-4181-b40f-867adf032c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:04.813794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f6bf8-275f-4181-b40f-867adf032c28 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:04.827342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 913/1823] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:34:04.831452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f6bf8-275f-4181-b40f-867adf032c28 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:04.841236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f6bf8-275f-4181-b40f-867adf032c28 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:04.845827 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a1a94-b85c-4357-ab43-338859056c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:04.846446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a1a94-b85c-4357-ab43-338859056c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:04.848696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a1a94-b85c-4357-ab43-338859056c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:04.848696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a1a94-b85c-4357-ab43-338859056c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:04.848696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a1a94-b85c-4357-ab43-338859056c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:04.849172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a1a94-b85c-4357-ab43-338859056c1c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:04.859315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 914/1824] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:04.863256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a1a94-b85c-4357-ab43-338859056c1c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:04.873417 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a1a94-b85c-4357-ab43-338859056c1c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:04.878452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa1980-de41-4854-a3f8-5850edbc042f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:04.879063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa1980-de41-4854-a3f8-5850edbc042f admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:04.879336 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa1980-de41-4854-a3f8-5850edbc042f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:04.879606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa1980-de41-4854-a3f8-5850edbc042f admin admin] PATH_INFO: `/v3/projects/7a51eb921f4f48fa9a12effdaa13f8c9/users/822513d2b1f3443383f01abdc485b264/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:04.893523 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa1980-de41-4854-a3f8-5850edbc042f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a51eb921f4f48fa9a12effdaa13f8c9, user_id=822513d2b1f3443383f01abdc485b264, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:04.896225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa1980-de41-4854-a3f8-5850edbc042f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:04.992472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 915/1825] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:04 2022] PUT /identity/v3/projects/7a51eb921f4f48fa9a12effdaa13f8c9/users/822513d2b1f3443383f01abdc485b264/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:05.003351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44b02d-8083-4a9c-b4b1-439ff20e6c99 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:05.003576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44b02d-8083-4a9c-b4b1-439ff20e6c99 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:05.003842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44b02d-8083-4a9c-b4b1-439ff20e6c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:05.036751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-7d44b02d-8083-4a9c-b4b1-439ff20e6c99 None None] MFA Rules not processed for user `822513d2b1f3443383f01abdc485b264`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:05.037940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7d44b02d-8083-4a9c-b4b1-439ff20e6c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:05.086766 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '907d456f9b4e404d8059bcacb6100d53'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:05.087587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '907d456f9b4e404d8059bcacb6100d53'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:05.095420 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '907d456f9b4e404d8059bcacb6100d53'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:05.103581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 916/1826] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:05.116282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44b02d-8083-4a9c-b4b1-439ff20e6c99 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:05.117657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7d44b02d-8083-4a9c-b4b1-439ff20e6c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:05.173197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44b02d-8083-4a9c-b4b1-439ff20e6c99 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:05.177059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae727f7-f7ce-4659-8444-5c7aea935be5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:05.177680 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae727f7-f7ce-4659-8444-5c7aea935be5 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:05.177942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae727f7-f7ce-4659-8444-5c7aea935be5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:05.178245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae727f7-f7ce-4659-8444-5c7aea935be5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:05.187104 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae727f7-f7ce-4659-8444-5c7aea935be5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:05.188531 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae727f7-f7ce-4659-8444-5c7aea935be5 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:05.192291 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '907d456f9b4e404d8059bcacb6100d53'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:05.196636 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '907d456f9b4e404d8059bcacb6100d53'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:05.202152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 917/1827] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:34:05.214551 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 911/1828] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:04 2022] DELETE /identity/v3/users/907d456f9b4e404d8059bcacb6100d53 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:05.219779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b41139-bc18-4436-a5d4-c4f68afa3d8d None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:05.220076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b41139-bc18-4436-a5d4-c4f68afa3d8d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:05.220349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b41139-bc18-4436-a5d4-c4f68afa3d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:05.224116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:05.225515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:05.240890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-49b41139-bc18-4436-a5d4-c4f68afa3d8d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:05.242554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-49b41139-bc18-4436-a5d4-c4f68afa3d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:05.291421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337f1f8-8d23-4c1f-b191-241fd82f4b1c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:05.296523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:05.297320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:05.297595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:05.297864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] PATH_INFO: `/v3/users/822513d2b1f3443383f01abdc485b264` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:05.299005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=822513d2b1f3443383f01abdc485b264)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:05.300340 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 918/1829] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:05.300943 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:05.311461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b41139-bc18-4436-a5d4-c4f68afa3d8d None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:05.312817 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-49b41139-bc18-4436-a5d4-c4f68afa3d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:05.329682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:05.374200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b41139-bc18-4436-a5d4-c4f68afa3d8d None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:05.378354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345f629-e469-4d04-9b51-252ba33fae7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:05.378966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345f629-e469-4d04-9b51-252ba33fae7d service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:05.379278 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345f629-e469-4d04-9b51-252ba33fae7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:05.379558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345f629-e469-4d04-9b51-252ba33fae7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:05.390073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345f629-e469-4d04-9b51-252ba33fae7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:05.391556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345f629-e469-4d04-9b51-252ba33fae7d service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:05.401176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 919/1830] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:34:05.561752 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '822513d2b1f3443383f01abdc485b264'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:05.562371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '822513d2b1f3443383f01abdc485b264'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:05.569967 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '822513d2b1f3443383f01abdc485b264'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:05.670160 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '822513d2b1f3443383f01abdc485b264'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:05.670863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '822513d2b1f3443383f01abdc485b264'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:05.751607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 912/1831] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:05 2022] DELETE /identity/v3/users/822513d2b1f3443383f01abdc485b264 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:05.757550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a84a4-1d3b-4927-a560-9dd341c002b0 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:05.758271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a84a4-1d3b-4927-a560-9dd341c002b0 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:05.763979 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a84a4-1d3b-4927-a560-9dd341c002b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:05.786320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:05.787016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:05.798960 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-5d4a84a4-1d3b-4927-a560-9dd341c002b0 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:05.804259 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5d4a84a4-1d3b-4927-a560-9dd341c002b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:05.890975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3029c3-b63a-447c-abfe-17902e484d19 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:05.904828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6261909-7822-4f7d-8961-ab9a6b451bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:05.905568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6261909-7822-4f7d-8961-ab9a6b451bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:05.905847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6261909-7822-4f7d-8961-ab9a6b451bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:05.906268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6261909-7822-4f7d-8961-ab9a6b451bd8 admin admin] PATH_INFO: `/v3/users/1a1067e91a1548bca2e70462d48b50d5` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:05.907479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6261909-7822-4f7d-8961-ab9a6b451bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a1067e91a1548bca2e70462d48b50d5)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:05.910718 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6261909-7822-4f7d-8961-ab9a6b451bd8 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:05.926933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 920/1832] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:05.937571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a84a4-1d3b-4927-a560-9dd341c002b0 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:05.939754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5d4a84a4-1d3b-4927-a560-9dd341c002b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:05.999183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:06.015036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a84a4-1d3b-4927-a560-9dd341c002b0 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:06.018591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786d83a-b07c-4031-abb8-de2a48111355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:06.019196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786d83a-b07c-4031-abb8-de2a48111355 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:06.019471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786d83a-b07c-4031-abb8-de2a48111355 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:06.019740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786d83a-b07c-4031-abb8-de2a48111355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:06.039727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786d83a-b07c-4031-abb8-de2a48111355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:06.039727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786d83a-b07c-4031-abb8-de2a48111355 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:06.056765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 921/1833] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:34:06.161827 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786d83a-b07c-4031-abb8-de2a48111355 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:06.164744 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8786d83a-b07c-4031-abb8-de2a48111355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:06.179651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786d83a-b07c-4031-abb8-de2a48111355 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:06.184897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:06.185739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:06.186022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:06.186396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] PATH_INFO: `/v3/projects/1c457a14369446769d55fbb2797e9efa` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:06.187726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c457a14369446769d55fbb2797e9efa)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:06.189785 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:06.215903 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b6261909-7822-4f7d-8961-ab9a6b451bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a1067e91a1548bca2e70462d48b50d5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.215903 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b6261909-7822-4f7d-8961-ab9a6b451bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a1067e91a1548bca2e70462d48b50d5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.223164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b6261909-7822-4f7d-8961-ab9a6b451bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a1067e91a1548bca2e70462d48b50d5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.370661 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b6261909-7822-4f7d-8961-ab9a6b451bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a1067e91a1548bca2e70462d48b50d5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.371307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b6261909-7822-4f7d-8961-ab9a6b451bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a1067e91a1548bca2e70462d48b50d5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.383409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c457a14369446769d55fbb2797e9efa', 'request_id': 'req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.393863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c457a14369446769d55fbb2797e9efa', 'request_id': 'req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.409117 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 913/1834] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:05 2022] DELETE /identity/v3/users/1a1067e91a1548bca2e70462d48b50d5 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:06.411252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ab9d6-e77c-4c8c-b9ad-8accba8ece3d None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:06.411546 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ab9d6-e77c-4c8c-b9ad-8accba8ece3d None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:06.411917 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ab9d6-e77c-4c8c-b9ad-8accba8ece3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:06.431942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-a81ab9d6-e77c-4c8c-b9ad-8accba8ece3d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:06.433157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a81ab9d6-e77c-4c8c-b9ad-8accba8ece3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:06.507499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 914/1835] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:06.518184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ab9d6-e77c-4c8c-b9ad-8accba8ece3d None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:06.519624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a81ab9d6-e77c-4c8c-b9ad-8accba8ece3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:06.552559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c457a14369446769d55fbb2797e9efa', 'request_id': 'req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.563708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] The token cache is being invalidate because project 1c457a14369446769d55fbb2797e9efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:06.564026 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.564679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.566288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 922/1836] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:06 2022] DELETE /identity/v3/projects/1c457a14369446769d55fbb2797e9efa => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:06.570086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:06.571381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:06.578056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ab9d6-e77c-4c8c-b9ad-8accba8ece3d None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:06.581722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bffc5-8c5c-44fd-aeea-4ce78d453d3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:06.582507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bffc5-8c5c-44fd-aeea-4ce78d453d3d service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:06.583354 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bffc5-8c5c-44fd-aeea-4ce78d453d3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:06.583354 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bffc5-8c5c-44fd-aeea-4ce78d453d3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:06.584965 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-491bffc5-8c5c-44fd-aeea-4ce78d453d3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:06.599313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bffc5-8c5c-44fd-aeea-4ce78d453d3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:06.601384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491bffc5-8c5c-44fd-aeea-4ce78d453d3d service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:06.614911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 915/1837] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:34:06.644697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc0db80-2baa-49eb-b3c3-39f675dd69d7 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:06.649811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:06.650479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:06.650759 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:06.651032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] PATH_INFO: `/v3/users/55ebea16e1a6440f9b9ce369b9e9f186` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:06.652235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ebea16e1a6440f9b9ce369b9e9f186)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:06.654839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:06.703549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bffc5-8c5c-44fd-aeea-4ce78d453d3d service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:06.710222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-491bffc5-8c5c-44fd-aeea-4ce78d453d3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:06.719807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bffc5-8c5c-44fd-aeea-4ce78d453d3d service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:06.725718 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:06.726400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:06.726701 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:06.726969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] PATH_INFO: `/v3/projects/7a51eb921f4f48fa9a12effdaa13f8c9` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:06.728478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a51eb921f4f48fa9a12effdaa13f8c9)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:06.730453 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:06.902797 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:06.961573 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a51eb921f4f48fa9a12effdaa13f8c9', 'request_id': 'req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.970463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a51eb921f4f48fa9a12effdaa13f8c9', 'request_id': 'req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.975578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ebea16e1a6440f9b9ce369b9e9f186'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:06.978118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ebea16e1a6440f9b9ce369b9e9f186'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.062035 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a51eb921f4f48fa9a12effdaa13f8c9', 'request_id': 'req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.080077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] The token cache is being invalidate because project 7a51eb921f4f48fa9a12effdaa13f8c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:07.085484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.085484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.085484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 916/1838] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:06 2022] DELETE /identity/v3/projects/7a51eb921f4f48fa9a12effdaa13f8c9 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:07.159013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ebea16e1a6440f9b9ce369b9e9f186'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.159838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ebea16e1a6440f9b9ce369b9e9f186'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.170912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:07.174375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ebea16e1a6440f9b9ce369b9e9f186'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.186889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:07.239995 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 923/1839] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:06 2022] DELETE /identity/v3/users/55ebea16e1a6440f9b9ce369b9e9f186 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:07.296997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:07.306855 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b090e5b-26ad-49c4-ba2c-ae092bcfb265 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:07.385127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:07.385877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:07.386194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:07.386555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] PATH_INFO: `/v3/projects/1f4ceae6c7344ab984deec36950db77d` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:07.389319 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4ceae6c7344ab984deec36950db77d)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:07.390405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:07.561836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4ceae6c7344ab984deec36950db77d', 'request_id': 'req-678933dc-0772-47cf-b2f7-92079c461363'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.569958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4ceae6c7344ab984deec36950db77d', 'request_id': 'req-678933dc-0772-47cf-b2f7-92079c461363'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.638945 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4ceae6c7344ab984deec36950db77d', 'request_id': 'req-678933dc-0772-47cf-b2f7-92079c461363'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.652571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] The token cache is being invalidate because project 1f4ceae6c7344ab984deec36950db77d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:07.652851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.654084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:07.655989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 917/1840] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:07 2022] DELETE /identity/v3/projects/1f4ceae6c7344ab984deec36950db77d => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:07.660092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:07.662397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:07.677507 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111a9a4-4534-4f09-be2c-0f42e4687520 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:07.741673 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:07.742343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:07.742622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:07.742894 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] PATH_INFO: `/v3/users/1e986543b1fc4218b0182d5f4204d72a` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:07.744005 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e986543b1fc4218b0182d5f4204d72a)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:07.745964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:07.841086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:07.844011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:07.866484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678933dc-0772-47cf-b2f7-92079c461363 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:07.872896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:07.873603 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:07.873950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:07.874327 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] PATH_INFO: `/v3/projects/07588c864f03445297da2a7c31754278` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:07.875610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07588c864f03445297da2a7c31754278)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:07.877810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:08.064539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07588c864f03445297da2a7c31754278', 'request_id': 'req-4d685898-1afa-4409-b3e8-954fd6ee8789'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.064825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e986543b1fc4218b0182d5f4204d72a'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.065475 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e986543b1fc4218b0182d5f4204d72a'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.071244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07588c864f03445297da2a7c31754278', 'request_id': 'req-4d685898-1afa-4409-b3e8-954fd6ee8789'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.140891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e986543b1fc4218b0182d5f4204d72a'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.141628 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e986543b1fc4218b0182d5f4204d72a'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.149618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e986543b1fc4218b0182d5f4204d72a'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.167265 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 924/1841] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:07 2022] DELETE /identity/v3/users/1e986543b1fc4218b0182d5f4204d72a => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:08.243995 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07588c864f03445297da2a7c31754278', 'request_id': 'req-4d685898-1afa-4409-b3e8-954fd6ee8789'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.263316 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] The token cache is being invalidate because project 07588c864f03445297da2a7c31754278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:08.263608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.264446 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.266412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 918/1842] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:07 2022] DELETE /identity/v3/projects/07588c864f03445297da2a7c31754278 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:08.270572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:08.273076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:08.340556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60480c35-04ca-4964-abfd-cb5a3fbdf4f4 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:08.362632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:08.365850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:08.366575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:08.373428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:08.373428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] PATH_INFO: `/v3/users/23d95a7e822146a683cf7a9156e01e16` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:08.373428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=23d95a7e822146a683cf7a9156e01e16)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:08.373428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:08.504007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:08.505520 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:08.574151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d685898-1afa-4409-b3e8-954fd6ee8789 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:08.579242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494190c-76e1-411f-a1ee-3a2f921cccc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:08.579872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494190c-76e1-411f-a1ee-3a2f921cccc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:08.580216 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494190c-76e1-411f-a1ee-3a2f921cccc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:08.580482 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494190c-76e1-411f-a1ee-3a2f921cccc3 admin admin] PATH_INFO: `/v3/users/378215912ecb4715adc67780c1faf9b6` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:08.581583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494190c-76e1-411f-a1ee-3a2f921cccc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=378215912ecb4715adc67780c1faf9b6)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:08.583496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494190c-76e1-411f-a1ee-3a2f921cccc3 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:08.724845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23d95a7e822146a683cf7a9156e01e16'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.726266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23d95a7e822146a683cf7a9156e01e16'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.874372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c494190c-76e1-411f-a1ee-3a2f921cccc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '378215912ecb4715adc67780c1faf9b6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.875020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c494190c-76e1-411f-a1ee-3a2f921cccc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '378215912ecb4715adc67780c1faf9b6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.882288 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c494190c-76e1-411f-a1ee-3a2f921cccc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '378215912ecb4715adc67780c1faf9b6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.940690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23d95a7e822146a683cf7a9156e01e16'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.941341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23d95a7e822146a683cf7a9156e01e16'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.948736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23d95a7e822146a683cf7a9156e01e16'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:08.966385 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 925/1843] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:08 2022] DELETE /identity/v3/users/23d95a7e822146a683cf7a9156e01e16 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:08.967836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:08.969216 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:09.039403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f8048-1a37-4b76-80e7-369532a330bd admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:09.049010 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:09.049010 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:09.049010 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:09.049010 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] PATH_INFO: `/v3/projects/43b37b5d497f453c9dbe8a94a926c312` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:09.049351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=43b37b5d497f453c9dbe8a94a926c312)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:09.050652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:09.077986 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c494190c-76e1-411f-a1ee-3a2f921cccc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '378215912ecb4715adc67780c1faf9b6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.078670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c494190c-76e1-411f-a1ee-3a2f921cccc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '378215912ecb4715adc67780c1faf9b6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.099493 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 919/1844] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:08 2022] DELETE /identity/v3/users/378215912ecb4715adc67780c1faf9b6 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:09.101841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f82529-845d-4ac0-9dc0-f2e427e25af4 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:09.102183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f82529-845d-4ac0-9dc0-f2e427e25af4 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:09.102440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f82529-845d-4ac0-9dc0-f2e427e25af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:09.122030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-60f82529-845d-4ac0-9dc0-f2e427e25af4 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:09.123260 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-60f82529-845d-4ac0-9dc0-f2e427e25af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:09.180801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 920/1845] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:09.191128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f82529-845d-4ac0-9dc0-f2e427e25af4 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:09.192524 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-60f82529-845d-4ac0-9dc0-f2e427e25af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:09.206374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43b37b5d497f453c9dbe8a94a926c312', 'request_id': 'req-671f873b-c154-4ea9-97c8-4fe159b3e3cf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.213434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43b37b5d497f453c9dbe8a94a926c312', 'request_id': 'req-671f873b-c154-4ea9-97c8-4fe159b3e3cf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.257416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f82529-845d-4ac0-9dc0-f2e427e25af4 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:09.300751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43b37b5d497f453c9dbe8a94a926c312', 'request_id': 'req-671f873b-c154-4ea9-97c8-4fe159b3e3cf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.307061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f4ee6-de4e-44d6-9359-ddef2eb3608a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:09.307686 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f4ee6-de4e-44d6-9359-ddef2eb3608a service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:09.307966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f4ee6-de4e-44d6-9359-ddef2eb3608a service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:09.308240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f4ee6-de4e-44d6-9359-ddef2eb3608a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:09.315516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] The token cache is being invalidate because project 43b37b5d497f453c9dbe8a94a926c312 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:09.315825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.316664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.321288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 926/1846] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:08 2022] DELETE /identity/v3/projects/43b37b5d497f453c9dbe8a94a926c312 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:09.322890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:09.325257 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:09.387899 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f4ee6-de4e-44d6-9359-ddef2eb3608a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:09.389531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f4ee6-de4e-44d6-9359-ddef2eb3608a service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:09.398238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f873b-c154-4ea9-97c8-4fe159b3e3cf admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:09.401177 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 921/1847] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:34:09.403821 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:09.404963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:09.404963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:09.404963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] PATH_INFO: `/v3/users/e7027ef062814ddf8818fcb02937499e` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:09.406201 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7027ef062814ddf8818fcb02937499e)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:09.408195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:09.450000 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f4ee6-de4e-44d6-9359-ddef2eb3608a service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:09.451486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-702f4ee6-de4e-44d6-9359-ddef2eb3608a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:09.466266 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f4ee6-de4e-44d6-9359-ddef2eb3608a service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:09.481195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:09.481658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:09.481945 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:09.482257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] PATH_INFO: `/v3/projects/678f1c0487034471ba012ac79b80a37f` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:09.483413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=678f1c0487034471ba012ac79b80a37f)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:09.485323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:09.787222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7027ef062814ddf8818fcb02937499e'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.788143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7027ef062814ddf8818fcb02937499e'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.792065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '678f1c0487034471ba012ac79b80a37f', 'request_id': 'req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.799249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '678f1c0487034471ba012ac79b80a37f', 'request_id': 'req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.855907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:09.992591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7027ef062814ddf8818fcb02937499e'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:09.993269 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7027ef062814ddf8818fcb02937499e'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:10.001413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7027ef062814ddf8818fcb02937499e'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:10.019446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 927/1848] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:09 2022] DELETE /identity/v3/users/e7027ef062814ddf8818fcb02937499e => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:10.053618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:10.055133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:10.107048 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '678f1c0487034471ba012ac79b80a37f', 'request_id': 'req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:10.120411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] The token cache is being invalidate because project 678f1c0487034471ba012ac79b80a37f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:10.120702 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:10.122668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e68c7157-ba65-48af-8d2c-7baf7b9d0dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:10.124387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 922/1849] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:09 2022] DELETE /identity/v3/projects/678f1c0487034471ba012ac79b80a37f => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:10.129218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c570-94fb-46ad-8284-224f1f4381fb None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.129495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c570-94fb-46ad-8284-224f1f4381fb None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.129879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248c570-94fb-46ad-8284-224f1f4381fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.156394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258859a-9219-4256-8e42-340eb9969306 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:10.156966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-f248c570-94fb-46ad-8284-224f1f4381fb None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:10.158855 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f248c570-94fb-46ad-8284-224f1f4381fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:10.161795 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:10.162586 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.163029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.163308 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] PATH_INFO: `/v3/projects/9d1af35950b44c53873fc2e47c4ae122` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.164480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d1af35950b44c53873fc2e47c4ae122)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:10.166462 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:10.169776 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 923/1850] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:10.181927 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c664-5005-4d45-81c2-325480188ea8 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.182587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c664-5005-4d45-81c2-325480188ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.183060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c664-5005-4d45-81c2-325480188ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.207336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-f954c664-5005-4d45-81c2-325480188ea8 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:10.208546 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f954c664-5005-4d45-81c2-325480188ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:10.216951 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 924/1851] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:10.221390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954c664-5005-4d45-81c2-325480188ea8 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:10.231230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954c664-5005-4d45-81c2-325480188ea8 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:10.235922 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e35b22-82ce-421f-819b-37b10760764d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:10.236559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35b22-82ce-421f-819b-37b10760764d admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.236836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35b22-82ce-421f-819b-37b10760764d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.237105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e35b22-82ce-421f-819b-37b10760764d admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.237750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e35b22-82ce-421f-819b-37b10760764d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:34:10.238098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e35b22-82ce-421f-819b-37b10760764d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:10.239653 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e35b22-82ce-421f-819b-37b10760764d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:10.262147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 925/1852] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:34:10.266033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e35b22-82ce-421f-819b-37b10760764d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:10.280207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e35b22-82ce-421f-819b-37b10760764d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:10.286758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7c8aa-a995-4a02-a58e-eb20bd942037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:10.287384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7c8aa-a995-4a02-a58e-eb20bd942037 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.287837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7c8aa-a995-4a02-a58e-eb20bd942037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.288286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7c8aa-a995-4a02-a58e-eb20bd942037 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.289635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7c8aa-a995-4a02-a58e-eb20bd942037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:10.294931 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7c8aa-a995-4a02-a58e-eb20bd942037 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:10.309991 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d1af35950b44c53873fc2e47c4ae122', 'request_id': 'req-d0e2de60-7a44-4230-9180-e4358cff74e2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:10.319792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d1af35950b44c53873fc2e47c4ae122', 'request_id': 'req-d0e2de60-7a44-4230-9180-e4358cff74e2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:10.420697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d1af35950b44c53873fc2e47c4ae122', 'request_id': 'req-d0e2de60-7a44-4230-9180-e4358cff74e2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:10.432604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 926/1853] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:10 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:10.434559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7c8aa-a995-4a02-a58e-eb20bd942037 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:10.438862 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] The token cache is being invalidate because project 9d1af35950b44c53873fc2e47c4ae122 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:10.439279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:10.440644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d0e2de60-7a44-4230-9180-e4358cff74e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:10.442699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 928/1854] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:10 2022] DELETE /identity/v3/projects/9d1af35950b44c53873fc2e47c4ae122 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:10.448920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ed31-2df4-4311-927a-8046bcb9007e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.449206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ed31-2df4-4311-927a-8046bcb9007e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.449548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ed31-2df4-4311-927a-8046bcb9007e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.471100 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-f253ed31-2df4-4311-927a-8046bcb9007e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:10.472302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f253ed31-2df4-4311-927a-8046bcb9007e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:10.504370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7c8aa-a995-4a02-a58e-eb20bd942037 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:10.509222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f653ee-ee7f-4f61-b25a-c5fa80f219e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:10.509840 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f653ee-ee7f-4f61-b25a-c5fa80f219e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.510165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f653ee-ee7f-4f61-b25a-c5fa80f219e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.510451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f653ee-ee7f-4f61-b25a-c5fa80f219e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.511229 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f653ee-ee7f-4f61-b25a-c5fa80f219e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:10.514292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f653ee-ee7f-4f61-b25a-c5fa80f219e8 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:10.541355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 929/1855] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:10.545628 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b944c-c6b5-4df9-8f1f-feb9feca4b35 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.545915 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b944c-c6b5-4df9-8f1f-feb9feca4b35 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.546221 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b944c-c6b5-4df9-8f1f-feb9feca4b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.566549 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-ed0b944c-c6b5-4df9-8f1f-feb9feca4b35 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:10.567761 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ed0b944c-c6b5-4df9-8f1f-feb9feca4b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:10.576171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 930/1856] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:10.580739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b944c-c6b5-4df9-8f1f-feb9feca4b35 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:10.589717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b944c-c6b5-4df9-8f1f-feb9feca4b35 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:10.594933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef2a29-ff4b-42ae-b813-619467c3b942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:10.595565 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef2a29-ff4b-42ae-b813-619467c3b942 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.595846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef2a29-ff4b-42ae-b813-619467c3b942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.596115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef2a29-ff4b-42ae-b813-619467c3b942 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.596755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef2a29-ff4b-42ae-b813-619467c3b942 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:34:10.597061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef2a29-ff4b-42ae-b813-619467c3b942 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:10.598695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef2a29-ff4b-42ae-b813-619467c3b942 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:10.612375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 931/1857] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:34:10.616711 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef2a29-ff4b-42ae-b813-619467c3b942 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:10.622086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 927/1858] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:10 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:34:10.626253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f653ee-ee7f-4f61-b25a-c5fa80f219e8 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:10.628212 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f8f653ee-ee7f-4f61-b25a-c5fa80f219e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:10.628785 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef2a29-ff4b-42ae-b813-619467c3b942 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:10.633815 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0577b-f724-4290-a0ca-2bffd55a466c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:10.634444 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0577b-f724-4290-a0ca-2bffd55a466c admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.634719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0577b-f724-4290-a0ca-2bffd55a466c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.634988 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0577b-f724-4290-a0ca-2bffd55a466c admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.635824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0577b-f724-4290-a0ca-2bffd55a466c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:10.639922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0577b-f724-4290-a0ca-2bffd55a466c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:10.641031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f653ee-ee7f-4f61-b25a-c5fa80f219e8 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:10.645770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f5809-1feb-4b1b-8b1a-2473276e985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:10.646478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f5809-1feb-4b1b-8b1a-2473276e985e admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.646826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f5809-1feb-4b1b-8b1a-2473276e985e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.647165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f5809-1feb-4b1b-8b1a-2473276e985e admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.648317 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f5809-1feb-4b1b-8b1a-2473276e985e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:10.650018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f5809-1feb-4b1b-8b1a-2473276e985e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:10.664646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 928/1859] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:34:10.665836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f5809-1feb-4b1b-8b1a-2473276e985e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:10.676861 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f5809-1feb-4b1b-8b1a-2473276e985e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:10.681947 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde00618-eaea-4b37-a58c-de0f6606a22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:10.682596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde00618-eaea-4b37-a58c-de0f6606a22f admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.682881 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde00618-eaea-4b37-a58c-de0f6606a22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.683166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde00618-eaea-4b37-a58c-de0f6606a22f admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.685166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde00618-eaea-4b37-a58c-de0f6606a22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:10.685437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde00618-eaea-4b37-a58c-de0f6606a22f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:10.699839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 929/1860] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:10.701531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde00618-eaea-4b37-a58c-de0f6606a22f admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:10.711792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde00618-eaea-4b37-a58c-de0f6606a22f admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:10.717029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e6226-7a12-4a59-b8a2-9bd1838dbca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:10.717684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e6226-7a12-4a59-b8a2-9bd1838dbca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.717966 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e6226-7a12-4a59-b8a2-9bd1838dbca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.718273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e6226-7a12-4a59-b8a2-9bd1838dbca1 admin admin] PATH_INFO: `/v3/projects/074d790a0eae445886b52d98550c442a/users/e1d3a82c17e0444c8f12dde70c827f6f/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.734154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e6226-7a12-4a59-b8a2-9bd1838dbca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=074d790a0eae445886b52d98550c442a, user_id=e1d3a82c17e0444c8f12dde70c827f6f, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:10.736933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e6226-7a12-4a59-b8a2-9bd1838dbca1 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:10.770215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 932/1861] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:10 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:34:10.773979 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0577b-f724-4290-a0ca-2bffd55a466c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:10.773979 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-25a0577b-f724-4290-a0ca-2bffd55a466c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:10.839430 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 930/1862] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:10 2022] PUT /identity/v3/projects/074d790a0eae445886b52d98550c442a/users/e1d3a82c17e0444c8f12dde70c827f6f/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:10.840996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e6226-7a12-4a59-b8a2-9bd1838dbca1 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:10.869446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0577b-f724-4290-a0ca-2bffd55a466c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:10.916017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e6226-7a12-4a59-b8a2-9bd1838dbca1 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:10.921769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c99ec-a53b-48e5-922f-46f7efb86b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:10.922431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c99ec-a53b-48e5-922f-46f7efb86b45 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.922706 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c99ec-a53b-48e5-922f-46f7efb86b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.922975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c99ec-a53b-48e5-922f-46f7efb86b45 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.923626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c99ec-a53b-48e5-922f-46f7efb86b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:10.925561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c99ec-a53b-48e5-922f-46f7efb86b45 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:10.938745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:10.941281 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:10.941570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:10.941850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] PATH_INFO: `/v3/projects/423f9941f3e44a779c73c8622d924c16` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:10.943053 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=423f9941f3e44a779c73c8622d924c16)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:10.944991 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.064929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 931/1863] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:10 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:11.064929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8b5ff-dd4f-4e5e-b6d6-5ddb14bcd72c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.064929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8b5ff-dd4f-4e5e-b6d6-5ddb14bcd72c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.064929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8b5ff-dd4f-4e5e-b6d6-5ddb14bcd72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.090654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-34f8b5ff-dd4f-4e5e-b6d6-5ddb14bcd72c None None] MFA Rules not processed for user `e1d3a82c17e0444c8f12dde70c827f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:11.091852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-34f8b5ff-dd4f-4e5e-b6d6-5ddb14bcd72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:11.119185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423f9941f3e44a779c73c8622d924c16', 'request_id': 'req-8e0d9e93-f545-46d3-9363-0d778b3f73a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:11.125670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423f9941f3e44a779c73c8622d924c16', 'request_id': 'req-8e0d9e93-f545-46d3-9363-0d778b3f73a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:11.161271 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 932/1864] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:11.162826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8b5ff-dd4f-4e5e-b6d6-5ddb14bcd72c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.173883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8b5ff-dd4f-4e5e-b6d6-5ddb14bcd72c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.192355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423f9941f3e44a779c73c8622d924c16', 'request_id': 'req-8e0d9e93-f545-46d3-9363-0d778b3f73a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:11.204698 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] The token cache is being invalidate because project 423f9941f3e44a779c73c8622d924c16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:11.204985 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:11.205621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:11.207367 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 933/1865] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:10 2022] DELETE /identity/v3/projects/423f9941f3e44a779c73c8622d924c16 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:11.208208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.210257 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:11.233654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e61daa-73c9-4de0-9791-dc3cc55330d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.234878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e61daa-73c9-4de0-9791-dc3cc55330d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.235178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e61daa-73c9-4de0-9791-dc3cc55330d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.235447 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e61daa-73c9-4de0-9791-dc3cc55330d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.236090 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e61daa-73c9-4de0-9791-dc3cc55330d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.237755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e61daa-73c9-4de0-9791-dc3cc55330d0 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.248455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 933/1866] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:11.249845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8331-2eb7-4e73-8faa-7406451cbbdd None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.250148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8331-2eb7-4e73-8faa-7406451cbbdd None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.250423 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e8331-2eb7-4e73-8faa-7406451cbbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.270577 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7e2e8331-2eb7-4e73-8faa-7406451cbbdd None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:11.271764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7e2e8331-2eb7-4e73-8faa-7406451cbbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:11.272081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d9e93-f545-46d3-9363-0d778b3f73a2 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.275962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9a6ca-8f7d-491a-bd6b-99dcbec951a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.276566 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9a6ca-8f7d-491a-bd6b-99dcbec951a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.276840 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9a6ca-8f7d-491a-bd6b-99dcbec951a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.277105 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9a6ca-8f7d-491a-bd6b-99dcbec951a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.278829 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-95f9a6ca-8f7d-491a-bd6b-99dcbec951a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:11.279712 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 934/1867] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:11.280473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e8331-2eb7-4e73-8faa-7406451cbbdd None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.281670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7e2e8331-2eb7-4e73-8faa-7406451cbbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:11.294775 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e8331-2eb7-4e73-8faa-7406451cbbdd None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.302561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bbe7c-203f-4e28-8ff6-96deea39bcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.303178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bbe7c-203f-4e28-8ff6-96deea39bcfb admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.303456 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bbe7c-203f-4e28-8ff6-96deea39bcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.303731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bbe7c-203f-4e28-8ff6-96deea39bcfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.304400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bbe7c-203f-4e28-8ff6-96deea39bcfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.306016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bbe7c-203f-4e28-8ff6-96deea39bcfb admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.327570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 935/1868] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:11.327570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ee9de-a9ff-4282-8aef-6822c9aec13b None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.327570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ee9de-a9ff-4282-8aef-6822c9aec13b None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.327570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ee9de-a9ff-4282-8aef-6822c9aec13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.341884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-6a5ee9de-a9ff-4282-8aef-6822c9aec13b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:11.343312 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6a5ee9de-a9ff-4282-8aef-6822c9aec13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:11.354624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 936/1869] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:11.355487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ee9de-a9ff-4282-8aef-6822c9aec13b None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.368841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9a6ca-8f7d-491a-bd6b-99dcbec951a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.370568 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9a6ca-8f7d-491a-bd6b-99dcbec951a7 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.370754 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ee9de-a9ff-4282-8aef-6822c9aec13b None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.376710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ba7cd-c451-49cf-9420-ab114e90a13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.377568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ba7cd-c451-49cf-9420-ab114e90a13b admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.377846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ba7cd-c451-49cf-9420-ab114e90a13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.378171 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ba7cd-c451-49cf-9420-ab114e90a13b admin admin] PATH_INFO: `/v3/projects/1af3bc15ae594a06a7508511ca040cf0/users/df6d651971b3435ca60797c9cf270bb7/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.384144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 934/1870] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:11.385402 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9a6ca-8f7d-491a-bd6b-99dcbec951a7 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.399935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9a6ca-8f7d-491a-bd6b-99dcbec951a7 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.400417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ba7cd-c451-49cf-9420-ab114e90a13b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af3bc15ae594a06a7508511ca040cf0, user_id=df6d651971b3435ca60797c9cf270bb7, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.403299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ba7cd-c451-49cf-9420-ab114e90a13b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.405839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947989a-842e-4715-a19d-1e0761ef3245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.407747 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947989a-842e-4715-a19d-1e0761ef3245 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.408138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947989a-842e-4715-a19d-1e0761ef3245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.408648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947989a-842e-4715-a19d-1e0761ef3245 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.409411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947989a-842e-4715-a19d-1e0761ef3245 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:34:11.410035 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947989a-842e-4715-a19d-1e0761ef3245 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.412183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947989a-842e-4715-a19d-1e0761ef3245 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.426079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 935/1871] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:34:11.430595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947989a-842e-4715-a19d-1e0761ef3245 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.444730 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947989a-842e-4715-a19d-1e0761ef3245 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.450495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718178e8-f034-4be9-a0a5-dbc40f4cd3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.451272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718178e8-f034-4be9-a0a5-dbc40f4cd3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.451544 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718178e8-f034-4be9-a0a5-dbc40f4cd3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.451813 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718178e8-f034-4be9-a0a5-dbc40f4cd3c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.452443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718178e8-f034-4be9-a0a5-dbc40f4cd3c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.454378 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718178e8-f034-4be9-a0a5-dbc40f4cd3c2 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.531155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 937/1872] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:11 2022] PUT /identity/v3/projects/1af3bc15ae594a06a7508511ca040cf0/users/df6d651971b3435ca60797c9cf270bb7/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:11.541812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f459797-79c6-4a98-9ab9-8d5c56cbffe9 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.542991 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f459797-79c6-4a98-9ab9-8d5c56cbffe9 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.543418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f459797-79c6-4a98-9ab9-8d5c56cbffe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.570834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 936/1873] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:11 2022] POST /identity/v3/projects => generated 363 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:11.575512 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718178e8-f034-4be9-a0a5-dbc40f4cd3c2 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.596934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7f459797-79c6-4a98-9ab9-8d5c56cbffe9 None None] MFA Rules not processed for user `df6d651971b3435ca60797c9cf270bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:11.596934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7f459797-79c6-4a98-9ab9-8d5c56cbffe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:11.635756 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718178e8-f034-4be9-a0a5-dbc40f4cd3c2 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.640811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418515fa-0c1a-4011-87d7-64e205ee6725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.641419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418515fa-0c1a-4011-87d7-64e205ee6725 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.641697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418515fa-0c1a-4011-87d7-64e205ee6725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.642774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418515fa-0c1a-4011-87d7-64e205ee6725 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.642871 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 938/1874] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:11.642961 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418515fa-0c1a-4011-87d7-64e205ee6725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.644854 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418515fa-0c1a-4011-87d7-64e205ee6725 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.646971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f459797-79c6-4a98-9ab9-8d5c56cbffe9 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.657724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f459797-79c6-4a98-9ab9-8d5c56cbffe9 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.662601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dbab9-bd2b-4f68-89fe-454af85bed64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.663198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dbab9-bd2b-4f68-89fe-454af85bed64 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.663476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dbab9-bd2b-4f68-89fe-454af85bed64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.663747 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dbab9-bd2b-4f68-89fe-454af85bed64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.664376 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52dbab9-bd2b-4f68-89fe-454af85bed64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.666283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52dbab9-bd2b-4f68-89fe-454af85bed64 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.672556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:11.735591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 937/1875] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:11 2022] POST /identity/v3/users => generated 282 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:34:11.739629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418515fa-0c1a-4011-87d7-64e205ee6725 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.749828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418515fa-0c1a-4011-87d7-64e205ee6725 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.754763 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22698b-f08a-406f-a6b2-1941830f4721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.755375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22698b-f08a-406f-a6b2-1941830f4721 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.755655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22698b-f08a-406f-a6b2-1941830f4721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.755928 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22698b-f08a-406f-a6b2-1941830f4721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.756688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22698b-f08a-406f-a6b2-1941830f4721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.758341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22698b-f08a-406f-a6b2-1941830f4721 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.768217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 938/1876] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:34:11.771877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22698b-f08a-406f-a6b2-1941830f4721 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.781206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22698b-f08a-406f-a6b2-1941830f4721 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.785742 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd838f-2b2f-49fa-bae5-f019c64ab1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.786337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd838f-2b2f-49fa-bae5-f019c64ab1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.786626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd838f-2b2f-49fa-bae5-f019c64ab1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.786898 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd838f-2b2f-49fa-bae5-f019c64ab1f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.787506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd838f-2b2f-49fa-bae5-f019c64ab1f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.789086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd838f-2b2f-49fa-bae5-f019c64ab1f7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.799718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 939/1877] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:34:11.803752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd838f-2b2f-49fa-bae5-f019c64ab1f7 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.813576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd838f-2b2f-49fa-bae5-f019c64ab1f7 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.818175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32930c4c-d2e5-4282-b4b5-10bbf45bc627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.818783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32930c4c-d2e5-4282-b4b5-10bbf45bc627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.819061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32930c4c-d2e5-4282-b4b5-10bbf45bc627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.819336 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32930c4c-d2e5-4282-b4b5-10bbf45bc627 admin admin] PATH_INFO: `/v3/projects/e8645c4c6d2149fcb30d5c57880edf65/users/5ef49baecf6541e5b7bc331bb133b34f/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.833362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32930c4c-d2e5-4282-b4b5-10bbf45bc627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8645c4c6d2149fcb30d5c57880edf65, user_id=5ef49baecf6541e5b7bc331bb133b34f, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.836037 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32930c4c-d2e5-4282-b4b5-10bbf45bc627 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.839850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 939/1878] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:11 2022] POST /identity/v3/projects => generated 379 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:34:11.843783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dbab9-bd2b-4f68-89fe-454af85bed64 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:11.903215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dbab9-bd2b-4f68-89fe-454af85bed64 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:11.908114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c922d-6136-4ce0-b55e-27f35a5886ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:11.908849 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c922d-6136-4ce0-b55e-27f35a5886ed admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.909191 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c922d-6136-4ce0-b55e-27f35a5886ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.909465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c922d-6136-4ce0-b55e-27f35a5886ed admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.910456 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c922d-6136-4ce0-b55e-27f35a5886ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:11.912185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c922d-6136-4ce0-b55e-27f35a5886ed admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:11.952615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 940/1879] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:11 2022] PUT /identity/v3/projects/e8645c4c6d2149fcb30d5c57880edf65/users/5ef49baecf6541e5b7bc331bb133b34f/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:11.963413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fdb70-d8c7-4f25-b8ff-6b3a63b81c19 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:11.963695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fdb70-d8c7-4f25-b8ff-6b3a63b81c19 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:11.963965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fdb70-d8c7-4f25-b8ff-6b3a63b81c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:11.994431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b45fdb70-d8c7-4f25-b8ff-6b3a63b81c19 None None] MFA Rules not processed for user `5ef49baecf6541e5b7bc331bb133b34f`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:11.995640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b45fdb70-d8c7-4f25-b8ff-6b3a63b81c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:12.044271 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 940/1880] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:11 2022] POST /identity/v3/users => generated 296 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:34:12.045901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c922d-6136-4ce0-b55e-27f35a5886ed admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:12.061970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c922d-6136-4ce0-b55e-27f35a5886ed admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:12.075908 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 941/1881] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:12.088701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fdb70-d8c7-4f25-b8ff-6b3a63b81c19 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:12.090940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b45fdb70-d8c7-4f25-b8ff-6b3a63b81c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:12.143017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287333b-75f2-41a3-b7d8-d193c48fcbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:12.143671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287333b-75f2-41a3-b7d8-d193c48fcbcb admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:12.143896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287333b-75f2-41a3-b7d8-d193c48fcbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:12.144168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287333b-75f2-41a3-b7d8-d193c48fcbcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:12.144859 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287333b-75f2-41a3-b7d8-d193c48fcbcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:12.146625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287333b-75f2-41a3-b7d8-d193c48fcbcb admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:12.160641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 941/1882] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:12.161002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fdb70-d8c7-4f25-b8ff-6b3a63b81c19 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:12.164905 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287333b-75f2-41a3-b7d8-d193c48fcbcb admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:12.165014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba78f3-2710-4ee4-a306-be830dacb415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:12.165369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba78f3-2710-4ee4-a306-be830dacb415 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:12.165644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba78f3-2710-4ee4-a306-be830dacb415 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:12.165997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba78f3-2710-4ee4-a306-be830dacb415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:12.175501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba78f3-2710-4ee4-a306-be830dacb415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:12.175779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287333b-75f2-41a3-b7d8-d193c48fcbcb admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:12.177134 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba78f3-2710-4ee4-a306-be830dacb415 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:12.180668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16bc39-d13f-4c50-b239-98a99ba00f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:12.181496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16bc39-d13f-4c50-b239-98a99ba00f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:12.181615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16bc39-d13f-4c50-b239-98a99ba00f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:12.181864 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16bc39-d13f-4c50-b239-98a99ba00f91 admin admin] PATH_INFO: `/v3/projects/8b332c38b1c248ccaf31fca44439322a/users/24b07d1434bc4eb3a53c738fdc69a03f/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:12.188406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 942/1883] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:34:12.199065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16bc39-d13f-4c50-b239-98a99ba00f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b332c38b1c248ccaf31fca44439322a, user_id=24b07d1434bc4eb3a53c738fdc69a03f, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:12.201797 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16bc39-d13f-4c50-b239-98a99ba00f91 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:12.299566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 942/1884] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:12 2022] PUT /identity/v3/projects/8b332c38b1c248ccaf31fca44439322a/users/24b07d1434bc4eb3a53c738fdc69a03f/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:12.300777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba78f3-2710-4ee4-a306-be830dacb415 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:12.312612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba78f3-2710-4ee4-a306-be830dacb415 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:12.379804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969833d5-68b4-48b8-b406-76f8f84222e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:12.380467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969833d5-68b4-48b8-b406-76f8f84222e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:12.380676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969833d5-68b4-48b8-b406-76f8f84222e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:12.380943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969833d5-68b4-48b8-b406-76f8f84222e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:12.381578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969833d5-68b4-48b8-b406-76f8f84222e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:12.383198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969833d5-68b4-48b8-b406-76f8f84222e7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:12.393445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 943/1885] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:12.397746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16bc39-d13f-4c50-b239-98a99ba00f91 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:12.408648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16bc39-d13f-4c50-b239-98a99ba00f91 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:12.416585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafa9b7-48ea-4527-a6fb-5d790262ebde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:12.420400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa9b7-48ea-4527-a6fb-5d790262ebde admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:12.420400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa9b7-48ea-4527-a6fb-5d790262ebde admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:12.420400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafa9b7-48ea-4527-a6fb-5d790262ebde admin admin] PATH_INFO: `/v3/projects/8b332c38b1c248ccaf31fca44439322a/users/24b07d1434bc4eb3a53c738fdc69a03f/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:12.420888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafa9b7-48ea-4527-a6fb-5d790262ebde admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b332c38b1c248ccaf31fca44439322a, user_id=24b07d1434bc4eb3a53c738fdc69a03f, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:12.423811 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cafa9b7-48ea-4527-a6fb-5d790262ebde admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:12.437469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 943/1886] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:12 2022] PUT /identity/v3/projects/8b332c38b1c248ccaf31fca44439322a/users/24b07d1434bc4eb3a53c738fdc69a03f/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:12.449632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3b170-60ac-4a7a-a1f8-57d38891d31c None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:12.449916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3b170-60ac-4a7a-a1f8-57d38891d31c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:12.450212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3b170-60ac-4a7a-a1f8-57d38891d31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:12.479275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-9ea3b170-60ac-4a7a-a1f8-57d38891d31c None None] MFA Rules not processed for user `24b07d1434bc4eb3a53c738fdc69a03f`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:12.480563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9ea3b170-60ac-4a7a-a1f8-57d38891d31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:12.547711 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 944/1887] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:12.564224 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafa9b7-48ea-4527-a6fb-5d790262ebde admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:12.589493 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:12.641476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cafa9b7-48ea-4527-a6fb-5d790262ebde admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:12.645349 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd96a9f-351d-454b-b4a2-5943ce0c93c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:12.645969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd96a9f-351d-454b-b4a2-5943ce0c93c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:12.646279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd96a9f-351d-454b-b4a2-5943ce0c93c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:12.646552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd96a9f-351d-454b-b4a2-5943ce0c93c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:12.657256 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd96a9f-351d-454b-b4a2-5943ce0c93c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:12.658657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd96a9f-351d-454b-b4a2-5943ce0c93c7 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:12.668896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 944/1888] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:13.979066 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3b170-60ac-4a7a-a1f8-57d38891d31c None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:13.987989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3b170-60ac-4a7a-a1f8-57d38891d31c None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:13.992543 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af817a-4cc0-40ec-a776-61544300393e tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24b07d1434bc4eb3a53c738fdc69a03f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b332c38b1c248ccaf31fca44439322a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:13.993138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af817a-4cc0-40ec-a776-61544300393e tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:13.993419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af817a-4cc0-40ec-a776-61544300393e tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:13.993691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af817a-4cc0-40ec-a776-61544300393e tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:13.994369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af817a-4cc0-40ec-a776-61544300393e tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:34:13.994677 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af817a-4cc0-40ec-a776-61544300393e tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:13.996300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af817a-4cc0-40ec-a776-61544300393e tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:14.014006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 945/1889] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:34:14.016852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd96a9f-351d-454b-b4a2-5943ce0c93c7 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:14.025764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd96a9f-351d-454b-b4a2-5943ce0c93c7 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:14.030994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ab33-c764-4a3c-88ba-5b2e30bffbe9 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24b07d1434bc4eb3a53c738fdc69a03f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b332c38b1c248ccaf31fca44439322a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:14.031588 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ab33-c764-4a3c-88ba-5b2e30bffbe9 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:14.031870 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ab33-c764-4a3c-88ba-5b2e30bffbe9 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:14.032142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781ab33-c764-4a3c-88ba-5b2e30bffbe9 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:14.034461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781ab33-c764-4a3c-88ba-5b2e30bffbe9 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:14.034688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3781ab33-c764-4a3c-88ba-5b2e30bffbe9 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:14.163067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 945/1890] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:14 2022] POST /identity/v3/projects => generated 354 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:14.164131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af817a-4cc0-40ec-a776-61544300393e tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:14.172685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:14.176782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af817a-4cc0-40ec-a776-61544300393e tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:14.240479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03df352-7b83-4233-8fac-50ea9bd243df tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24b07d1434bc4eb3a53c738fdc69a03f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b332c38b1c248ccaf31fca44439322a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:14.241097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03df352-7b83-4233-8fac-50ea9bd243df tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:14.241383 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03df352-7b83-4233-8fac-50ea9bd243df tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:14.241661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03df352-7b83-4233-8fac-50ea9bd243df tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:14.242741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03df352-7b83-4233-8fac-50ea9bd243df tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:34:14.243374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03df352-7b83-4233-8fac-50ea9bd243df tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:14.245253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03df352-7b83-4233-8fac-50ea9bd243df tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:14.263541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 946/1891] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:34:14.300393 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ab33-c764-4a3c-88ba-5b2e30bffbe9 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:14.313960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781ab33-c764-4a3c-88ba-5b2e30bffbe9 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:14.319653 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24b07d1434bc4eb3a53c738fdc69a03f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b332c38b1c248ccaf31fca44439322a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:14.320290 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:14.320568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:14.320842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] PATH_INFO: `/v3/projects/cdb8afa60f7843f8aaea5a3c31697a1b` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:14.321955 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cdb8afa60f7843f8aaea5a3c31697a1b)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:14.324077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:14.457628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdb8afa60f7843f8aaea5a3c31697a1b', 'request_id': 'req-05b49a51-49c3-4da2-8759-a09eb280cd73'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:14.464066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdb8afa60f7843f8aaea5a3c31697a1b', 'request_id': 'req-05b49a51-49c3-4da2-8759-a09eb280cd73'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:14.580656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdb8afa60f7843f8aaea5a3c31697a1b', 'request_id': 'req-05b49a51-49c3-4da2-8759-a09eb280cd73'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:14.591535 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] The token cache is being invalidate because project cdb8afa60f7843f8aaea5a3c31697a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:14.591830 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:14.592471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:14.593978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 946/1892] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:14 2022] DELETE /identity/v3/projects/cdb8afa60f7843f8aaea5a3c31697a1b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:15.029831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:15.189552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03df352-7b83-4233-8fac-50ea9bd243df tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:15.195920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c03df352-7b83-4233-8fac-50ea9bd243df tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:15.263833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03df352-7b83-4233-8fac-50ea9bd243df tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:15.318839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda2f57-59b6-4d1a-bf15-e04cd30c458a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:15.319466 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda2f57-59b6-4d1a-bf15-e04cd30c458a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:15.319740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda2f57-59b6-4d1a-bf15-e04cd30c458a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:15.320023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda2f57-59b6-4d1a-bf15-e04cd30c458a admin admin] PATH_INFO: `/v3/users/e1d3a82c17e0444c8f12dde70c827f6f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:15.321160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda2f57-59b6-4d1a-bf15-e04cd30c458a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d3a82c17e0444c8f12dde70c827f6f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:15.323174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda2f57-59b6-4d1a-bf15-e04cd30c458a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:15.398452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:15.399894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:15.413843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b49a51-49c3-4da2-8759-a09eb280cd73 tempest-VolumeQuotaClassesTest-798488744 tempest-VolumeQuotaClassesTest-798488744-project-admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:15.419180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f064354-bc70-47af-829b-b53ad1b4a1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:15.419727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064354-bc70-47af-829b-b53ad1b4a1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:15.420060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064354-bc70-47af-829b-b53ad1b4a1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:15.420336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f064354-bc70-47af-829b-b53ad1b4a1ad admin admin] PATH_INFO: `/v3/users/df6d651971b3435ca60797c9cf270bb7` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:15.421618 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f064354-bc70-47af-829b-b53ad1b4a1ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=df6d651971b3435ca60797c9cf270bb7)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:15.423574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f064354-bc70-47af-829b-b53ad1b4a1ad admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:15.587857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ffda2f57-59b6-4d1a-bf15-e04cd30c458a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d3a82c17e0444c8f12dde70c827f6f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:15.588504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ffda2f57-59b6-4d1a-bf15-e04cd30c458a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d3a82c17e0444c8f12dde70c827f6f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:15.692638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4f064354-bc70-47af-829b-b53ad1b4a1ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df6d651971b3435ca60797c9cf270bb7'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:15.693293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4f064354-bc70-47af-829b-b53ad1b4a1ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df6d651971b3435ca60797c9cf270bb7'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:15.700323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4f064354-bc70-47af-829b-b53ad1b4a1ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df6d651971b3435ca60797c9cf270bb7'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:15.712033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ffda2f57-59b6-4d1a-bf15-e04cd30c458a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d3a82c17e0444c8f12dde70c827f6f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:15.712663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ffda2f57-59b6-4d1a-bf15-e04cd30c458a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d3a82c17e0444c8f12dde70c827f6f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:15.720380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ffda2f57-59b6-4d1a-bf15-e04cd30c458a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d3a82c17e0444c8f12dde70c827f6f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:15.737943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 947/1893] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:15 2022] DELETE /identity/v3/users/e1d3a82c17e0444c8f12dde70c827f6f => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:15.742203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce67d4b-bf62-4e50-afbf-88b91117b1eb None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:15.749352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce67d4b-bf62-4e50-afbf-88b91117b1eb None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:15.749352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce67d4b-bf62-4e50-afbf-88b91117b1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:15.751591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:15.763004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-7ce67d4b-bf62-4e50-afbf-88b91117b1eb None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:15.764212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7ce67d4b-bf62-4e50-afbf-88b91117b1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:15.815065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4f064354-bc70-47af-829b-b53ad1b4a1ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df6d651971b3435ca60797c9cf270bb7'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:15.815762 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-4f064354-bc70-47af-829b-b53ad1b4a1ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df6d651971b3435ca60797c9cf270bb7'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:15.822062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 948/1894] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:15.832303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce67d4b-bf62-4e50-afbf-88b91117b1eb None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:15.833845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7ce67d4b-bf62-4e50-afbf-88b91117b1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:15.836676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 947/1895] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:15 2022] DELETE /identity/v3/users/df6d651971b3435ca60797c9cf270bb7 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:15.841587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e1609-5b52-4227-9bca-7f25829ec861 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:15.841869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e1609-5b52-4227-9bca-7f25829ec861 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:15.842170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e1609-5b52-4227-9bca-7f25829ec861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:15.862015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-b42e1609-5b52-4227-9bca-7f25829ec861 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:15.863236 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b42e1609-5b52-4227-9bca-7f25829ec861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:15.892558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce67d4b-bf62-4e50-afbf-88b91117b1eb None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:15.931511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 948/1896] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:15.947311 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e2d6a-4be5-4f1f-806e-80c0d305f846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:15.947919 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e2d6a-4be5-4f1f-806e-80c0d305f846 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:15.948030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e1609-5b52-4227-9bca-7f25829ec861 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:15.948337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e2d6a-4be5-4f1f-806e-80c0d305f846 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:15.948615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e2d6a-4be5-4f1f-806e-80c0d305f846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:15.958472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e1609-5b52-4227-9bca-7f25829ec861 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:15.959858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e2d6a-4be5-4f1f-806e-80c0d305f846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:15.961754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e2d6a-4be5-4f1f-806e-80c0d305f846 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:15.962325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe363-e08e-4b96-9551-42f8ff0fe59e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:15.962930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe363-e08e-4b96-9551-42f8ff0fe59e service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:15.963207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe363-e08e-4b96-9551-42f8ff0fe59e service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:15.963476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16fe363-e08e-4b96-9551-42f8ff0fe59e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:15.973382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 949/1897] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:34:15.974351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fe363-e08e-4b96-9551-42f8ff0fe59e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:15.976034 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16fe363-e08e-4b96-9551-42f8ff0fe59e service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:15.987621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 949/1898] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:34:16.392712 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:16.577132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e2d6a-4be5-4f1f-806e-80c0d305f846 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:16.578585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-115e2d6a-4be5-4f1f-806e-80c0d305f846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:16.590913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e2d6a-4be5-4f1f-806e-80c0d305f846 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:16.596279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:16.596910 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:16.597229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:16.597533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] PATH_INFO: `/v3/projects/1af3bc15ae594a06a7508511ca040cf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:16.598658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1af3bc15ae594a06a7508511ca040cf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:16.601284 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:16.617613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe363-e08e-4b96-9551-42f8ff0fe59e service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:16.619958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b16fe363-e08e-4b96-9551-42f8ff0fe59e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:16.637831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16fe363-e08e-4b96-9551-42f8ff0fe59e service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:16.642554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:16.643368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:16.643655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:16.643927 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] PATH_INFO: `/v3/projects/074d790a0eae445886b52d98550c442a` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:16.645078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=074d790a0eae445886b52d98550c442a)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:16.647051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:16.924374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1af3bc15ae594a06a7508511ca040cf0', 'request_id': 'req-0317fcb4-0d57-4020-927a-2f3318e55e49'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:16.930524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1af3bc15ae594a06a7508511ca040cf0', 'request_id': 'req-0317fcb4-0d57-4020-927a-2f3318e55e49'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.035284 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '074d790a0eae445886b52d98550c442a', 'request_id': 'req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.042011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '074d790a0eae445886b52d98550c442a', 'request_id': 'req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.135302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1af3bc15ae594a06a7508511ca040cf0', 'request_id': 'req-0317fcb4-0d57-4020-927a-2f3318e55e49'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.146118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] The token cache is being invalidate because project 1af3bc15ae594a06a7508511ca040cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:17.146408 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.147030 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.148508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 950/1899] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:16 2022] DELETE /identity/v3/projects/1af3bc15ae594a06a7508511ca040cf0 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:17.152267 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:17.153544 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:17.216290 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317fcb4-0d57-4020-927a-2f3318e55e49 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:17.273984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:17.274635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:17.274909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:17.275177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] PATH_INFO: `/v3/users/24b07d1434bc4eb3a53c738fdc69a03f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:17.276393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b07d1434bc4eb3a53c738fdc69a03f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:17.278856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:17.357665 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '074d790a0eae445886b52d98550c442a', 'request_id': 'req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.368748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] The token cache is being invalidate because project 074d790a0eae445886b52d98550c442a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:17.369035 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.369668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-18a3cff5-b8b0-4754-a882-c14cc81f6b2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.371214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 950/1900] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:16 2022] DELETE /identity/v3/projects/074d790a0eae445886b52d98550c442a => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:17.375789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a206d35-d01d-4939-bba3-42e56e94fbb3 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:17.376067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a206d35-d01d-4939-bba3-42e56e94fbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:17.376336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a206d35-d01d-4939-bba3-42e56e94fbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:17.396394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2a206d35-d01d-4939-bba3-42e56e94fbb3 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:17.397585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2a206d35-d01d-4939-bba3-42e56e94fbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:17.406212 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 951/1901] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:17.414006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dde10-c664-4c7b-a075-0142c609d862 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:17.414309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dde10-c664-4c7b-a075-0142c609d862 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:17.414600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dde10-c664-4c7b-a075-0142c609d862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:17.435436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-521dde10-c664-4c7b-a075-0142c609d862 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:17.436688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-521dde10-c664-4c7b-a075-0142c609d862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:17.444974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 952/1902] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:17.449212 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dde10-c664-4c7b-a075-0142c609d862 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:17.460108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dde10-c664-4c7b-a075-0142c609d862 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:17.465261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0ce4e-85a3-4bf7-b5c8-7f106ec31aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:17.465863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0ce4e-85a3-4bf7-b5c8-7f106ec31aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:17.466172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0ce4e-85a3-4bf7-b5c8-7f106ec31aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:17.466439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0ce4e-85a3-4bf7-b5c8-7f106ec31aa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:17.467079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0ce4e-85a3-4bf7-b5c8-7f106ec31aa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:34:17.467380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0ce4e-85a3-4bf7-b5c8-7f106ec31aa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:17.468957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0ce4e-85a3-4bf7-b5c8-7f106ec31aa9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:17.484511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 953/1903] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:34:17.487187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b07d1434bc4eb3a53c738fdc69a03f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.488040 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b07d1434bc4eb3a53c738fdc69a03f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.489495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0ce4e-85a3-4bf7-b5c8-7f106ec31aa9 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:17.493318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-67d0ce4e-85a3-4bf7-b5c8-7f106ec31aa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:17.506842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0ce4e-85a3-4bf7-b5c8-7f106ec31aa9 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:17.511672 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47eeb53-6976-4cf4-8952-f6e239bc4543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:17.512293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eeb53-6976-4cf4-8952-f6e239bc4543 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:17.512573 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eeb53-6976-4cf4-8952-f6e239bc4543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:17.512848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47eeb53-6976-4cf4-8952-f6e239bc4543 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:17.513563 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47eeb53-6976-4cf4-8952-f6e239bc4543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:17.515875 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47eeb53-6976-4cf4-8952-f6e239bc4543 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:17.563683 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b07d1434bc4eb3a53c738fdc69a03f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.564468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b07d1434bc4eb3a53c738fdc69a03f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.573131 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b07d1434bc4eb3a53c738fdc69a03f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:17.597343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 951/1904] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:17 2022] DELETE /identity/v3/users/24b07d1434bc4eb3a53c738fdc69a03f => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:17.607814 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 954/1905] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:17 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:34:17.609852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:17.682417 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce7240-b51e-49d5-9a29-6612a2f31c57 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:17.739027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b094a-caff-4a86-94f9-24793e52f154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:17.739664 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b094a-caff-4a86-94f9-24793e52f154 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:17.739941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b094a-caff-4a86-94f9-24793e52f154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:17.740210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b094a-caff-4a86-94f9-24793e52f154 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:17.740849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b094a-caff-4a86-94f9-24793e52f154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:17.742786 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b094a-caff-4a86-94f9-24793e52f154 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:17.850397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:17.898748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 952/1906] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:17 2022] POST /identity/v3/users => generated 288 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:17.906345 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47eeb53-6976-4cf4-8952-f6e239bc4543 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:17.929370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47eeb53-6976-4cf4-8952-f6e239bc4543 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:17.949826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f6b9f-caaf-43a8-8ed3-bbe1cf716875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:17.958933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f6b9f-caaf-43a8-8ed3-bbe1cf716875 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:17.959432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f6b9f-caaf-43a8-8ed3-bbe1cf716875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:17.959813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01f6b9f-caaf-43a8-8ed3-bbe1cf716875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:17.960648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f6b9f-caaf-43a8-8ed3-bbe1cf716875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:17.962554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01f6b9f-caaf-43a8-8ed3-bbe1cf716875 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:17.975051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 955/1907] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:17.979843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b094a-caff-4a86-94f9-24793e52f154 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:17.990499 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b094a-caff-4a86-94f9-24793e52f154 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:17.995337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95067e9b-c093-4872-92b1-abbef9944117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:17.996560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95067e9b-c093-4872-92b1-abbef9944117 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:17.997059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95067e9b-c093-4872-92b1-abbef9944117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:17.997407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95067e9b-c093-4872-92b1-abbef9944117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:17.998771 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95067e9b-c093-4872-92b1-abbef9944117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:18.000461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95067e9b-c093-4872-92b1-abbef9944117 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:18.015785 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 953/1908] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:34:18.020586 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f6b9f-caaf-43a8-8ed3-bbe1cf716875 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:18.027262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01f6b9f-caaf-43a8-8ed3-bbe1cf716875 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:18.031889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74d356-c960-4b54-ab6f-3e196bc3b324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:18.032557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74d356-c960-4b54-ab6f-3e196bc3b324 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:18.032832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74d356-c960-4b54-ab6f-3e196bc3b324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:18.033303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74d356-c960-4b54-ab6f-3e196bc3b324 admin admin] PATH_INFO: `/v3/projects/7daf87a0028b4f58a2c69d933579233b/users/8b0f7380468140229b0bb3adf9e9034c/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:18.048100 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74d356-c960-4b54-ab6f-3e196bc3b324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7daf87a0028b4f58a2c69d933579233b, user_id=8b0f7380468140229b0bb3adf9e9034c, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:18.050775 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74d356-c960-4b54-ab6f-3e196bc3b324 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:18.165449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95067e9b-c093-4872-92b1-abbef9944117 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:18.166926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 956/1909] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:18 2022] PUT /identity/v3/projects/7daf87a0028b4f58a2c69d933579233b/users/8b0f7380468140229b0bb3adf9e9034c/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:18.167808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-95067e9b-c093-4872-92b1-abbef9944117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:18.177044 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c6109-9afb-4175-9a24-972517c81ed1 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:18.177334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c6109-9afb-4175-9a24-972517c81ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:18.177609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c6109-9afb-4175-9a24-972517c81ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:18.183919 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95067e9b-c093-4872-92b1-abbef9944117 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:18.213158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9b1c6109-9afb-4175-9a24-972517c81ed1 None None] MFA Rules not processed for user `8b0f7380468140229b0bb3adf9e9034c`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:18.214293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9b1c6109-9afb-4175-9a24-972517c81ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:18.255826 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:18.256541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:18.256824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:18.257235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] PATH_INFO: `/v3/projects/8b332c38b1c248ccaf31fca44439322a` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:18.258493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b332c38b1c248ccaf31fca44439322a)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:18.260424 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:18.310835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 957/1910] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:18.375481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b332c38b1c248ccaf31fca44439322a', 'request_id': 'req-7038eecf-8960-43a7-aa2d-fe5c0a342c52'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:18.382873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b332c38b1c248ccaf31fca44439322a', 'request_id': 'req-7038eecf-8960-43a7-aa2d-fe5c0a342c52'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:18.466640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b332c38b1c248ccaf31fca44439322a', 'request_id': 'req-7038eecf-8960-43a7-aa2d-fe5c0a342c52'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:18.467380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:18.479574 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] The token cache is being invalidate because project 8b332c38b1c248ccaf31fca44439322a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:18.480560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:18.481393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-7038eecf-8960-43a7-aa2d-fe5c0a342c52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:18.483540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 954/1911] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:18 2022] DELETE /identity/v3/projects/8b332c38b1c248ccaf31fca44439322a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:18.488028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c6109-9afb-4175-9a24-972517c81ed1 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:18.489822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9b1c6109-9afb-4175-9a24-972517c81ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:18.491103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6ea3c-e8e7-4d11-b14e-28f6eea7d2b0 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:18.491395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6ea3c-e8e7-4d11-b14e-28f6eea7d2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:18.491679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6ea3c-e8e7-4d11-b14e-28f6eea7d2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:18.513186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-92d6ea3c-e8e7-4d11-b14e-28f6eea7d2b0 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:18.514437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-92d6ea3c-e8e7-4d11-b14e-28f6eea7d2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:18.554700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c6109-9afb-4175-9a24-972517c81ed1 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:18.574823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 955/1912] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:18.582615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e2dc0-330f-44d5-a772-655d036583ac None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:18.582784 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e2dc0-330f-44d5-a772-655d036583ac None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:18.583017 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e2dc0-330f-44d5-a772-655d036583ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:18.608604 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-6b8e2dc0-330f-44d5-a772-655d036583ac None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:18.609836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6b8e2dc0-330f-44d5-a772-655d036583ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:18.611078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ee2b1-73cb-435a-a3a4-b553857afaf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:18.611800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee2b1-73cb-435a-a3a4-b553857afaf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:18.612202 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee2b1-73cb-435a-a3a4-b553857afaf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:18.612297 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee2b1-73cb-435a-a3a4-b553857afaf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:18.614068 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-dd0ee2b1-73cb-435a-a3a4-b553857afaf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:18.620072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 956/1913] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:18.624832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e2dc0-330f-44d5-a772-655d036583ac None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:18.635425 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e2dc0-330f-44d5-a772-655d036583ac None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:18.641168 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ee3a5-69a6-469e-9fca-714828a1588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:18.642011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ee3a5-69a6-469e-9fca-714828a1588d admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:18.642333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ee3a5-69a6-469e-9fca-714828a1588d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:18.642606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ee3a5-69a6-469e-9fca-714828a1588d admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:18.643260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ee3a5-69a6-469e-9fca-714828a1588d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:34:18.643559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ee3a5-69a6-469e-9fca-714828a1588d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:18.645157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ee3a5-69a6-469e-9fca-714828a1588d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:18.661128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 957/1914] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:34:18.662228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ee3a5-69a6-469e-9fca-714828a1588d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:18.675224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ee3a5-69a6-469e-9fca-714828a1588d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:18.682575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67796b1b-a9d4-4257-a1d4-55269182f567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:18.683874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67796b1b-a9d4-4257-a1d4-55269182f567 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:18.684212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67796b1b-a9d4-4257-a1d4-55269182f567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:18.684548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67796b1b-a9d4-4257-a1d4-55269182f567 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:18.685244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67796b1b-a9d4-4257-a1d4-55269182f567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:18.687292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67796b1b-a9d4-4257-a1d4-55269182f567 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:18.707413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ee2b1-73cb-435a-a3a4-b553857afaf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:18.709055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ee2b1-73cb-435a-a3a4-b553857afaf6 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:18.719761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 958/1915] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:18.816498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:18.836857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 958/1916] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:18 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:34:18.841367 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ee2b1-73cb-435a-a3a4-b553857afaf6 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:18.852771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ee2b1-73cb-435a-a3a4-b553857afaf6 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:18.912327 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a5369-3b06-43ce-bd29-8887ba48497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:18.912934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a5369-3b06-43ce-bd29-8887ba48497e admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:18.913209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a5369-3b06-43ce-bd29-8887ba48497e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:18.913481 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a5369-3b06-43ce-bd29-8887ba48497e admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:18.914161 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a5369-3b06-43ce-bd29-8887ba48497e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:18.916110 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a5369-3b06-43ce-bd29-8887ba48497e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:19.026021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 959/1917] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:18 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:19.030225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67796b1b-a9d4-4257-a1d4-55269182f567 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:19.048111 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67796b1b-a9d4-4257-a1d4-55269182f567 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:19.055792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd096582-8d74-49d7-b807-5e5c23dd8e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:19.056398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd096582-8d74-49d7-b807-5e5c23dd8e5c admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:19.056687 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd096582-8d74-49d7-b807-5e5c23dd8e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:19.056933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd096582-8d74-49d7-b807-5e5c23dd8e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:19.057569 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd096582-8d74-49d7-b807-5e5c23dd8e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:19.059844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd096582-8d74-49d7-b807-5e5c23dd8e5c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:19.061520 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:19.073794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 959/1918] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:19.075337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a5369-3b06-43ce-bd29-8887ba48497e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:19.086214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a5369-3b06-43ce-bd29-8887ba48497e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:19.093058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff46ce-3c90-4c66-aa68-90f844aa5c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:19.093058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff46ce-3c90-4c66-aa68-90f844aa5c81 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:19.093058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff46ce-3c90-4c66-aa68-90f844aa5c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:19.093058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff46ce-3c90-4c66-aa68-90f844aa5c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:19.093623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff46ce-3c90-4c66-aa68-90f844aa5c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:19.095339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff46ce-3c90-4c66-aa68-90f844aa5c81 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:19.115467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 960/1919] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:34:19.122508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd096582-8d74-49d7-b807-5e5c23dd8e5c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:19.133205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd096582-8d74-49d7-b807-5e5c23dd8e5c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:19.137693 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73855f3-365a-4e48-93d2-731b00e3aafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:19.138382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73855f3-365a-4e48-93d2-731b00e3aafe admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:19.138657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73855f3-365a-4e48-93d2-731b00e3aafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:19.138932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73855f3-365a-4e48-93d2-731b00e3aafe admin admin] PATH_INFO: `/v3/projects/d70750ba404b4c8399bb3775be381378/users/e5567fcedd9748269aa75cef7b99e812/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:19.154912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73855f3-365a-4e48-93d2-731b00e3aafe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d70750ba404b4c8399bb3775be381378, user_id=e5567fcedd9748269aa75cef7b99e812, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:19.158008 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73855f3-365a-4e48-93d2-731b00e3aafe admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:19.260487 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 960/1920] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:19 2022] PUT /identity/v3/projects/d70750ba404b4c8399bb3775be381378/users/e5567fcedd9748269aa75cef7b99e812/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:19.272156 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71db3-fd6b-4a19-a9a0-da9a9001aa9b None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:19.272516 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71db3-fd6b-4a19-a9a0-da9a9001aa9b None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:19.273667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71db3-fd6b-4a19-a9a0-da9a9001aa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:19.305880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-edf71db3-fd6b-4a19-a9a0-da9a9001aa9b None None] MFA Rules not processed for user `e5567fcedd9748269aa75cef7b99e812`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:19.305880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-edf71db3-fd6b-4a19-a9a0-da9a9001aa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:19.365737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 961/1921] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:19.370218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73855f3-365a-4e48-93d2-731b00e3aafe admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:19.431076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73855f3-365a-4e48-93d2-731b00e3aafe admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:19.436339 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ad6df-299d-4203-9594-7f00e180733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:19.436957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ad6df-299d-4203-9594-7f00e180733e admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:19.437226 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ad6df-299d-4203-9594-7f00e180733e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:19.437516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ad6df-299d-4203-9594-7f00e180733e admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:19.438291 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ad6df-299d-4203-9594-7f00e180733e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:19.440197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ad6df-299d-4203-9594-7f00e180733e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:19.559522 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 961/1922] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:19 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:19.563259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf71db3-fd6b-4a19-a9a0-da9a9001aa9b None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:19.576921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf71db3-fd6b-4a19-a9a0-da9a9001aa9b None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:19.635754 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09793ac8-8ba1-48f8-9a73-a7ac603dab44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:19.636505 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09793ac8-8ba1-48f8-9a73-a7ac603dab44 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:19.636791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09793ac8-8ba1-48f8-9a73-a7ac603dab44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:19.637070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09793ac8-8ba1-48f8-9a73-a7ac603dab44 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:19.637725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09793ac8-8ba1-48f8-9a73-a7ac603dab44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:19.639691 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09793ac8-8ba1-48f8-9a73-a7ac603dab44 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:19.723581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:19.786280 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 962/1923] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:19 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:19.790373 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ad6df-299d-4203-9594-7f00e180733e admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:19.801032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ad6df-299d-4203-9594-7f00e180733e admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:19.805476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd240b0-ffcf-40fe-9fb8-23f3c1ed52ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:19.806098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd240b0-ffcf-40fe-9fb8-23f3c1ed52ec admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:19.806383 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd240b0-ffcf-40fe-9fb8-23f3c1ed52ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:19.806649 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd240b0-ffcf-40fe-9fb8-23f3c1ed52ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:19.807365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd240b0-ffcf-40fe-9fb8-23f3c1ed52ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:19.811306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd240b0-ffcf-40fe-9fb8-23f3c1ed52ec admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:19.822691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 962/1924] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:19.827030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09793ac8-8ba1-48f8-9a73-a7ac603dab44 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:19.837112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09793ac8-8ba1-48f8-9a73-a7ac603dab44 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:19.842851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72605d-7ef9-46cc-985b-965da47942bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:19.843904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72605d-7ef9-46cc-985b-965da47942bc admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:19.843904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72605d-7ef9-46cc-985b-965da47942bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:19.844072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72605d-7ef9-46cc-985b-965da47942bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:19.844621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72605d-7ef9-46cc-985b-965da47942bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:19.846247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72605d-7ef9-46cc-985b-965da47942bc admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:19.856723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 963/1925] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:19.860981 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd240b0-ffcf-40fe-9fb8-23f3c1ed52ec admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:19.871985 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd240b0-ffcf-40fe-9fb8-23f3c1ed52ec admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:19.882106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580fa4b-a581-4527-8d4c-8e0a4f148703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:19.882781 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580fa4b-a581-4527-8d4c-8e0a4f148703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:19.883054 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580fa4b-a581-4527-8d4c-8e0a4f148703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:19.883327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580fa4b-a581-4527-8d4c-8e0a4f148703 admin admin] PATH_INFO: `/v3/projects/5efc2fceaae144b8bac446397b07e849/users/de79bc41bd9744eba6d85c2525a9e5e9/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:19.899597 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580fa4b-a581-4527-8d4c-8e0a4f148703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5efc2fceaae144b8bac446397b07e849, user_id=de79bc41bd9744eba6d85c2525a9e5e9, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:19.902936 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580fa4b-a581-4527-8d4c-8e0a4f148703 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:20.027876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:20.034537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 963/1926] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:19 2022] PUT /identity/v3/projects/5efc2fceaae144b8bac446397b07e849/users/de79bc41bd9744eba6d85c2525a9e5e9/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:20.046025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a6d7d-84bc-41a7-8f88-6ef1c50a54ae None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:20.046351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a6d7d-84bc-41a7-8f88-6ef1c50a54ae None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:20.046656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a6d7d-84bc-41a7-8f88-6ef1c50a54ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:20.074489 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-217a6d7d-84bc-41a7-8f88-6ef1c50a54ae None None] MFA Rules not processed for user `de79bc41bd9744eba6d85c2525a9e5e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:20.075698 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-217a6d7d-84bc-41a7-8f88-6ef1c50a54ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:20.145934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 964/1927] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:20.153588 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580fa4b-a581-4527-8d4c-8e0a4f148703 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:20.217642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580fa4b-a581-4527-8d4c-8e0a4f148703 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:20.222451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194c954-c146-4cca-87ad-fce70f80a9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:20.223061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194c954-c146-4cca-87ad-fce70f80a9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:20.223368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194c954-c146-4cca-87ad-fce70f80a9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:20.223641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194c954-c146-4cca-87ad-fce70f80a9c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:20.224278 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194c954-c146-4cca-87ad-fce70f80a9c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:20.226206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194c954-c146-4cca-87ad-fce70f80a9c1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:20.341720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 964/1928] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:20 2022] POST /identity/v3/projects => generated 387 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:20.345679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a6d7d-84bc-41a7-8f88-6ef1c50a54ae None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:20.356173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a6d7d-84bc-41a7-8f88-6ef1c50a54ae None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:20.411322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd03f0-8246-4600-bde6-0885c6c6abf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:20.411932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd03f0-8246-4600-bde6-0885c6c6abf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:20.412212 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd03f0-8246-4600-bde6-0885c6c6abf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:20.412486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd03f0-8246-4600-bde6-0885c6c6abf6 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:20.413143 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd03f0-8246-4600-bde6-0885c6c6abf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:20.415083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd03f0-8246-4600-bde6-0885c6c6abf6 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:20.551897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 965/1929] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:20 2022] POST /identity/v3/users => generated 300 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:20.555942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194c954-c146-4cca-87ad-fce70f80a9c1 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:20.557631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:INSERT=3,keystone:SELECT=83,keystone:DELETE=14 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:20.566760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194c954-c146-4cca-87ad-fce70f80a9c1 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:20.571324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda4c27-c880-4cd1-89ee-561c48760655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:20.571923 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda4c27-c880-4cd1-89ee-561c48760655 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:20.572199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda4c27-c880-4cd1-89ee-561c48760655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:20.572467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda4c27-c880-4cd1-89ee-561c48760655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:20.573103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda4c27-c880-4cd1-89ee-561c48760655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:20.575282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda4c27-c880-4cd1-89ee-561c48760655 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:20.585694 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 965/1930] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:34:20.589759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd03f0-8246-4600-bde6-0885c6c6abf6 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:20.600396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd03f0-8246-4600-bde6-0885c6c6abf6 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:20.605509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95205a39-8c55-45ae-85b3-69319b4b4ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:20.606182 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95205a39-8c55-45ae-85b3-69319b4b4ea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:20.606463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95205a39-8c55-45ae-85b3-69319b4b4ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:20.606742 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95205a39-8c55-45ae-85b3-69319b4b4ea4 admin admin] PATH_INFO: `/v3/projects/c8cd5e338a764f2391e593770ce63581/users/624bfed3d7eb407098092f74a5f92fef/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:20.621593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95205a39-8c55-45ae-85b3-69319b4b4ea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8cd5e338a764f2391e593770ce63581, user_id=624bfed3d7eb407098092f74a5f92fef, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:20.624250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95205a39-8c55-45ae-85b3-69319b4b4ea4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:20.678102 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:20.769205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 966/1931] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:20 2022] PUT /identity/v3/projects/c8cd5e338a764f2391e593770ce63581/users/624bfed3d7eb407098092f74a5f92fef/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:20.773063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda4c27-c880-4cd1-89ee-561c48760655 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:20.782998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda4c27-c880-4cd1-89ee-561c48760655 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:20.839978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21940f-1c4e-4464-ac5c-9d6c019aa2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:20.840631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21940f-1c4e-4464-ac5c-9d6c019aa2ef admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:20.840859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21940f-1c4e-4464-ac5c-9d6c019aa2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:20.841145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21940f-1c4e-4464-ac5c-9d6c019aa2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:20.841779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21940f-1c4e-4464-ac5c-9d6c019aa2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:20.843405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21940f-1c4e-4464-ac5c-9d6c019aa2ef admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:20.854351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 966/1932] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:20.858074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95205a39-8c55-45ae-85b3-69319b4b4ea4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:20.868119 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95205a39-8c55-45ae-85b3-69319b4b4ea4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:20.873378 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e11d4-2b6b-4222-bc35-ceb33a714d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:20.874018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e11d4-2b6b-4222-bc35-ceb33a714d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:20.874325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e11d4-2b6b-4222-bc35-ceb33a714d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:20.874803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e11d4-2b6b-4222-bc35-ceb33a714d30 admin admin] PATH_INFO: `/v3/projects/c8cd5e338a764f2391e593770ce63581/users/624bfed3d7eb407098092f74a5f92fef/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:20.876500 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e11d4-2b6b-4222-bc35-ceb33a714d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8cd5e338a764f2391e593770ce63581, user_id=624bfed3d7eb407098092f74a5f92fef, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:20.879422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e11d4-2b6b-4222-bc35-ceb33a714d30 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:20.889419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 967/1933] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:20 2022] PUT /identity/v3/projects/c8cd5e338a764f2391e593770ce63581/users/624bfed3d7eb407098092f74a5f92fef/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:20.899658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0405aa9-bbe5-43b9-99ab-ff0e030abc06 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:20.899943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0405aa9-bbe5-43b9-99ab-ff0e030abc06 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:20.900219 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0405aa9-bbe5-43b9-99ab-ff0e030abc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:20.928094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-f0405aa9-bbe5-43b9-99ab-ff0e030abc06 None None] MFA Rules not processed for user `624bfed3d7eb407098092f74a5f92fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:20.929285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f0405aa9-bbe5-43b9-99ab-ff0e030abc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:20.994844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 967/1934] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:21.005737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e11d4-2b6b-4222-bc35-ceb33a714d30 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:21.008253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-315e11d4-2b6b-4222-bc35-ceb33a714d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:21.009185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:21.073629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e11d4-2b6b-4222-bc35-ceb33a714d30 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:21.077476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdaa8f-b603-481c-8756-2cbf5351ce1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:21.078194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdaa8f-b603-481c-8756-2cbf5351ce1a service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:21.078476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdaa8f-b603-481c-8756-2cbf5351ce1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:21.080348 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdaa8f-b603-481c-8756-2cbf5351ce1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:21.087889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fdaa8f-b603-481c-8756-2cbf5351ce1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:21.089862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fdaa8f-b603-481c-8756-2cbf5351ce1a service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:21.102660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 968/1935] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:21.135445 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:21.249446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:21.431772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:21.786538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:21.866401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:22.215001 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:22.277855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:22.906603 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:23.067366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550bf7e-1f85-4235-b709-ecac8bab0d2b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:23.067726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550bf7e-1f85-4235-b709-ecac8bab0d2b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:23.067962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550bf7e-1f85-4235-b709-ecac8bab0d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:23.090751 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8550bf7e-1f85-4235-b709-ecac8bab0d2b None None] MFA Rules not processed for user `8b0f7380468140229b0bb3adf9e9034c`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:23.092036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8550bf7e-1f85-4235-b709-ecac8bab0d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:23.158215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 968/1936] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:23.192484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c74ffd-b90e-489f-a121-e127c6cfcd9c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:23.192484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c74ffd-b90e-489f-a121-e127c6cfcd9c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:23.192484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c74ffd-b90e-489f-a121-e127c6cfcd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:23.233217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:23.240025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-17c74ffd-b90e-489f-a121-e127c6cfcd9c None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:23.242131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-17c74ffd-b90e-489f-a121-e127c6cfcd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:23.309641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 969/1937] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:34:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:23.360626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550bf7e-1f85-4235-b709-ecac8bab0d2b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:23.362159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8550bf7e-1f85-4235-b709-ecac8bab0d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:23.493878 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550bf7e-1f85-4235-b709-ecac8bab0d2b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:23.512948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a386-bd48-4137-b3d7-a31b35ed114d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:23.514103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c74ffd-b90e-489f-a121-e127c6cfcd9c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:23.514543 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a386-bd48-4137-b3d7-a31b35ed114d service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:23.514820 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a386-bd48-4137-b3d7-a31b35ed114d service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:23.515091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a386-bd48-4137-b3d7-a31b35ed114d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:23.516457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-17c74ffd-b90e-489f-a121-e127c6cfcd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:23.524520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a386-bd48-4137-b3d7-a31b35ed114d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:23.526264 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a386-bd48-4137-b3d7-a31b35ed114d service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:23.536452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c74ffd-b90e-489f-a121-e127c6cfcd9c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:23.542226 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaaab95-bc4e-408f-9a64-60ed5ea37f70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:23.543254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaaab95-bc4e-408f-9a64-60ed5ea37f70 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:23.543685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaaab95-bc4e-408f-9a64-60ed5ea37f70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:23.544101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaaab95-bc4e-408f-9a64-60ed5ea37f70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:23.545029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 969/1938] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:34:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:34:23.629439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaaab95-bc4e-408f-9a64-60ed5ea37f70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:23.637865 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaaab95-bc4e-408f-9a64-60ed5ea37f70 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:23.653560 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 970/1939] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:33.533604 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=83 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:33.647410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:33.849110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a386-bd48-4137-b3d7-a31b35ed114d service swift] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:33.868851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a386-bd48-4137-b3d7-a31b35ed114d service swift] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:33.883017 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e5ad8-7566-4559-947e-72653e0d7550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:33.883951 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e5ad8-7566-4559-947e-72653e0d7550 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:33.884437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e5ad8-7566-4559-947e-72653e0d7550 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:33.884866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e5ad8-7566-4559-947e-72653e0d7550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:34.004345 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e5ad8-7566-4559-947e-72653e0d7550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:34.006133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e5ad8-7566-4559-947e-72653e0d7550 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:34.023477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 970/1940] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:35.337774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77094b-2d1d-4bf9-b91a-f9017fa82a14 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:35.338341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77094b-2d1d-4bf9-b91a-f9017fa82a14 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:35.338341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77094b-2d1d-4bf9-b91a-f9017fa82a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:35.362541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-6c77094b-2d1d-4bf9-b91a-f9017fa82a14 None None] MFA Rules not processed for user `75e5de4fccd243e580a12e2475162451`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:35.364114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6c77094b-2d1d-4bf9-b91a-f9017fa82a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:35.442424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 971/1941] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:39.667985 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e5ad8-7566-4559-947e-72653e0d7550 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:39.838553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e5ad8-7566-4559-947e-72653e0d7550 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:39.846760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:39.847603 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:39.848269 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:39.848701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] PATH_INFO: `/v3/users/e5567fcedd9748269aa75cef7b99e812` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:39.850366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5567fcedd9748269aa75cef7b99e812)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:39.852807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:40.199608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5567fcedd9748269aa75cef7b99e812'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:40.199608 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5567fcedd9748269aa75cef7b99e812'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:40.333796 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5567fcedd9748269aa75cef7b99e812'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:40.334641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5567fcedd9748269aa75cef7b99e812'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:40.355932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5567fcedd9748269aa75cef7b99e812'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:40.402889 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 971/1942] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:39 2022] DELETE /identity/v3/users/e5567fcedd9748269aa75cef7b99e812 => generated 0 bytes in 733 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:40.406490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ce89c-d442-4f34-b52f-8568dfab3aee None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:40.411459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ce89c-d442-4f34-b52f-8568dfab3aee None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:40.411459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ce89c-d442-4f34-b52f-8568dfab3aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:40.459909 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-479ce89c-d442-4f34-b52f-8568dfab3aee None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:40.461497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-479ce89c-d442-4f34-b52f-8568dfab3aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:40.677531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 972/1943] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:40.687272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:40.690974 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:40.724172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:40.776116 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dda8d9-e874-4990-97a3-4e39e5688314 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:40.780717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6697-0fb0-45a4-b19c-d5146bfb0857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:40.781631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6697-0fb0-45a4-b19c-d5146bfb0857 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:40.782034 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6697-0fb0-45a4-b19c-d5146bfb0857 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:40.782488 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b6697-0fb0-45a4-b19c-d5146bfb0857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:40.798236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b6697-0fb0-45a4-b19c-d5146bfb0857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:40.799864 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525b6697-0fb0-45a4-b19c-d5146bfb0857 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:40.822137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 972/1944] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:42.197279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ce89c-d442-4f34-b52f-8568dfab3aee None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:42.201275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-479ce89c-d442-4f34-b52f-8568dfab3aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:42.239372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ce89c-d442-4f34-b52f-8568dfab3aee None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:42.253875 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:42.258096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:42.258096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:42.258096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] PATH_INFO: `/v3/projects/d70750ba404b4c8399bb3775be381378` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:42.267557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d70750ba404b4c8399bb3775be381378)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:42.270609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:42.547733 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd70750ba404b4c8399bb3775be381378', 'request_id': 'req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:42.567484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd70750ba404b4c8399bb3775be381378', 'request_id': 'req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:42.687162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd70750ba404b4c8399bb3775be381378', 'request_id': 'req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:42.721549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] The token cache is being invalidate because project d70750ba404b4c8399bb3775be381378 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:42.724381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:42.726964 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c800a2d9-e2dd-48ec-a441-b6ee4d4f008f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:42.731954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 973/1945] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:42 2022] DELETE /identity/v3/projects/d70750ba404b4c8399bb3775be381378 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:42.735792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6697-0fb0-45a4-b19c-d5146bfb0857 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:42.752453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b6697-0fb0-45a4-b19c-d5146bfb0857 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:42.924506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:42.924506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:42.924506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:42.924506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] PATH_INFO: `/v3/users/de79bc41bd9744eba6d85c2525a9e5e9` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:42.929486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de79bc41bd9744eba6d85c2525a9e5e9)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:42.943632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:43.199288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de79bc41bd9744eba6d85c2525a9e5e9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:43.201072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de79bc41bd9744eba6d85c2525a9e5e9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:43.307090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de79bc41bd9744eba6d85c2525a9e5e9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:43.307450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de79bc41bd9744eba6d85c2525a9e5e9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:43.320223 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de79bc41bd9744eba6d85c2525a9e5e9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:43.370850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 973/1946] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:42 2022] DELETE /identity/v3/users/de79bc41bd9744eba6d85c2525a9e5e9 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:44.608505 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0aae2-b05d-4637-ab02-6de1e8222f28 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:44.608968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0aae2-b05d-4637-ab02-6de1e8222f28 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:44.609250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0aae2-b05d-4637-ab02-6de1e8222f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:44.629368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:44.631360 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:44.688066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-6ee0aae2-b05d-4637-ab02-6de1e8222f28 None None] MFA Rules not processed for user `9f0fad4e9be748c786fe114895d8970c`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:44.695836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6ee0aae2-b05d-4637-ab02-6de1e8222f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:44.735714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:44.827449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:44.834949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e1945-0ab5-4f77-96cb-bacf7a60da56 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:44.853132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:44.853798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:44.854097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:44.854375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] PATH_INFO: `/v3/projects/5efc2fceaae144b8bac446397b07e849` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:44.857354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5efc2fceaae144b8bac446397b07e849)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:44.860957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:44.912357 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 974/1947] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:45.030871 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5efc2fceaae144b8bac446397b07e849', 'request_id': 'req-a093837c-2f4f-4330-8859-7c7b2ff93729'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:45.066320 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5efc2fceaae144b8bac446397b07e849', 'request_id': 'req-a093837c-2f4f-4330-8859-7c7b2ff93729'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:45.155876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5efc2fceaae144b8bac446397b07e849', 'request_id': 'req-a093837c-2f4f-4330-8859-7c7b2ff93729'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:45.171063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] The token cache is being invalidate because project 5efc2fceaae144b8bac446397b07e849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:45.171569 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:45.172302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:45.176777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 974/1948] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:44 2022] DELETE /identity/v3/projects/5efc2fceaae144b8bac446397b07e849 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:45.178461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0aae2-b05d-4637-ab02-6de1e8222f28 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:45.191406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0aae2-b05d-4637-ab02-6de1e8222f28 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:45.272177 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014022e4-3efb-49db-82b2-f46165cd947b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:45.279239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014022e4-3efb-49db-82b2-f46165cd947b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:45.279239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014022e4-3efb-49db-82b2-f46165cd947b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:45.279239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014022e4-3efb-49db-82b2-f46165cd947b admin admin] PATH_INFO: `/v3/users/624bfed3d7eb407098092f74a5f92fef` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:45.279239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014022e4-3efb-49db-82b2-f46165cd947b admin admin] RBAC: Authorizing `identity:delete_user(user_id=624bfed3d7eb407098092f74a5f92fef)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:45.279239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014022e4-3efb-49db-82b2-f46165cd947b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:45.482020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-014022e4-3efb-49db-82b2-f46165cd947b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624bfed3d7eb407098092f74a5f92fef'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:45.482886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-014022e4-3efb-49db-82b2-f46165cd947b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624bfed3d7eb407098092f74a5f92fef'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:45.491268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-014022e4-3efb-49db-82b2-f46165cd947b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624bfed3d7eb407098092f74a5f92fef'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:45.562564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-014022e4-3efb-49db-82b2-f46165cd947b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624bfed3d7eb407098092f74a5f92fef'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:45.563491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-014022e4-3efb-49db-82b2-f46165cd947b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624bfed3d7eb407098092f74a5f92fef'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:45.610554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 975/1949] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:45 2022] DELETE /identity/v3/users/624bfed3d7eb407098092f74a5f92fef => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:46.407525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:46.409551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:46.427498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093837c-2f4f-4330-8859-7c7b2ff93729 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:46.596604 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:46.597915 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:46.598248 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:46.598539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] PATH_INFO: `/v3/projects/c8cd5e338a764f2391e593770ce63581` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:46.600056 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8cd5e338a764f2391e593770ce63581)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:46.602794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:46.800484 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8cd5e338a764f2391e593770ce63581', 'request_id': 'req-ff07160e-69b5-4835-a097-fc92d0bf4153'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:46.812671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8cd5e338a764f2391e593770ce63581', 'request_id': 'req-ff07160e-69b5-4835-a097-fc92d0bf4153'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:46.922883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8cd5e338a764f2391e593770ce63581', 'request_id': 'req-ff07160e-69b5-4835-a097-fc92d0bf4153'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:46.942210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:46.952428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] The token cache is being invalidate because project c8cd5e338a764f2391e593770ce63581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:46.952805 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:46.953912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ff07160e-69b5-4835-a097-fc92d0bf4153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:46.960386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 975/1950] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:46 2022] DELETE /identity/v3/projects/c8cd5e338a764f2391e593770ce63581 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:46.965363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbe16e-9722-4c4e-9648-595afc03c441 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:46.965363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbe16e-9722-4c4e-9648-595afc03c441 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:46.965363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbe16e-9722-4c4e-9648-595afc03c441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:46.987171 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-63bbe16e-9722-4c4e-9648-595afc03c441 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:46.988590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-63bbe16e-9722-4c4e-9648-595afc03c441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:47.069656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:47.094221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 976/1951] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:47.099071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72354-4e13-4ef1-b610-3ec864f91853 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:47.099733 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72354-4e13-4ef1-b610-3ec864f91853 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:47.100400 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72354-4e13-4ef1-b610-3ec864f91853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:47.128791 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-84d72354-4e13-4ef1-b610-3ec864f91853 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:47.130036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-84d72354-4e13-4ef1-b610-3ec864f91853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:47.139108 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 976/1952] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:47.143362 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbe16e-9722-4c4e-9648-595afc03c441 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:47.155146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bbe16e-9722-4c4e-9648-595afc03c441 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:47.160476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b62a51-1bf2-4ef2-8713-c7824bd09f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:47.165216 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b62a51-1bf2-4ef2-8713-c7824bd09f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:47.165216 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b62a51-1bf2-4ef2-8713-c7824bd09f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:47.165216 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b62a51-1bf2-4ef2-8713-c7824bd09f12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:47.165216 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b62a51-1bf2-4ef2-8713-c7824bd09f12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:34:47.165216 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b62a51-1bf2-4ef2-8713-c7824bd09f12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:47.165216 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b62a51-1bf2-4ef2-8713-c7824bd09f12 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:47.178229 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 977/1953] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:34:47.183006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d72354-4e13-4ef1-b610-3ec864f91853 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:47.197598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d72354-4e13-4ef1-b610-3ec864f91853 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:47.205801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf40241-14f5-494c-870f-82cf28fc1178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:47.206973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf40241-14f5-494c-870f-82cf28fc1178 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:47.207203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf40241-14f5-494c-870f-82cf28fc1178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:47.207485 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf40241-14f5-494c-870f-82cf28fc1178 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:47.208660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf40241-14f5-494c-870f-82cf28fc1178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:47.212611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf40241-14f5-494c-870f-82cf28fc1178 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:47.313172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 977/1954] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:47 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:47.322971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b62a51-1bf2-4ef2-8713-c7824bd09f12 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:47.334458 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b62a51-1bf2-4ef2-8713-c7824bd09f12 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:47.414958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39befefa-0cfe-4945-83eb-17ca5455d505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:47.415585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39befefa-0cfe-4945-83eb-17ca5455d505 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:47.415867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39befefa-0cfe-4945-83eb-17ca5455d505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:47.416146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39befefa-0cfe-4945-83eb-17ca5455d505 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:47.416958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39befefa-0cfe-4945-83eb-17ca5455d505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:47.419499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39befefa-0cfe-4945-83eb-17ca5455d505 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:47.538470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 978/1955] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:47 2022] POST /identity/v3/users => generated 294 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:47.542740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf40241-14f5-494c-870f-82cf28fc1178 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:47.553651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf40241-14f5-494c-870f-82cf28fc1178 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:47.558219 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc328c-2cd5-477a-8197-209437858926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:47.558832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc328c-2cd5-477a-8197-209437858926 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:47.559195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc328c-2cd5-477a-8197-209437858926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:47.559471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc328c-2cd5-477a-8197-209437858926 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:47.560143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc328c-2cd5-477a-8197-209437858926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:47.562230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc328c-2cd5-477a-8197-209437858926 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:47.573932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 978/1956] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:47.578408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39befefa-0cfe-4945-83eb-17ca5455d505 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:47.589542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39befefa-0cfe-4945-83eb-17ca5455d505 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:47.607183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a76881-fdfd-4a05-855e-34d0b24a5fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:47.607183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76881-fdfd-4a05-855e-34d0b24a5fab admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:47.607183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76881-fdfd-4a05-855e-34d0b24a5fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:47.607183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a76881-fdfd-4a05-855e-34d0b24a5fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:47.607689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a76881-fdfd-4a05-855e-34d0b24a5fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:47.607689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a76881-fdfd-4a05-855e-34d0b24a5fab admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:47.611606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 979/1957] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:47.615301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc328c-2cd5-477a-8197-209437858926 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:47.626714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc328c-2cd5-477a-8197-209437858926 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:47.632408 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33797c-3a91-4fad-bc59-06985f831dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:47.633104 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33797c-3a91-4fad-bc59-06985f831dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:47.633335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33797c-3a91-4fad-bc59-06985f831dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:47.633638 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33797c-3a91-4fad-bc59-06985f831dd7 admin admin] PATH_INFO: `/v3/projects/576f1b90a8674ff294e6da2bc303a0d2/users/35542fae464945a89b39f57dca4194a9/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:47.649193 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad33797c-3a91-4fad-bc59-06985f831dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=576f1b90a8674ff294e6da2bc303a0d2, user_id=35542fae464945a89b39f57dca4194a9, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:47.651861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad33797c-3a91-4fad-bc59-06985f831dd7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:47.780101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 979/1958] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:47 2022] PUT /identity/v3/projects/576f1b90a8674ff294e6da2bc303a0d2/users/35542fae464945a89b39f57dca4194a9/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:47.791450 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bad0e4-e792-4385-afff-d2bc2c7105c4 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:47.791822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bad0e4-e792-4385-afff-d2bc2c7105c4 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:47.792109 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bad0e4-e792-4385-afff-d2bc2c7105c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:47.820747 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-98bad0e4-e792-4385-afff-d2bc2c7105c4 None None] MFA Rules not processed for user `35542fae464945a89b39f57dca4194a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:47.821938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-98bad0e4-e792-4385-afff-d2bc2c7105c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:47.898127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 980/1959] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:47.899333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33797c-3a91-4fad-bc59-06985f831dd7 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:47.969847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33797c-3a91-4fad-bc59-06985f831dd7 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:47.974988 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc7334-e558-4c83-b795-e7e62170794f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:47.975662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc7334-e558-4c83-b795-e7e62170794f admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:47.975968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc7334-e558-4c83-b795-e7e62170794f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:47.976660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc7334-e558-4c83-b795-e7e62170794f admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:47.977386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc7334-e558-4c83-b795-e7e62170794f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:47.979493 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc7334-e558-4c83-b795-e7e62170794f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:48.088559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 980/1960] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:47 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:34:48.089717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bad0e4-e792-4385-afff-d2bc2c7105c4 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:48.112936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bad0e4-e792-4385-afff-d2bc2c7105c4 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:48.166562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:48.179259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24580511-1a60-4725-b8fc-f7d2ff4884e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:48.179878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24580511-1a60-4725-b8fc-f7d2ff4884e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:48.180153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24580511-1a60-4725-b8fc-f7d2ff4884e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:48.180417 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24580511-1a60-4725-b8fc-f7d2ff4884e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:48.181056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24580511-1a60-4725-b8fc-f7d2ff4884e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:48.182998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24580511-1a60-4725-b8fc-f7d2ff4884e5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:48.277753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 981/1961] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:48 2022] POST /identity/v3/users => generated 300 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:34:48.282385 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc7334-e558-4c83-b795-e7e62170794f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:48.292286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc7334-e558-4c83-b795-e7e62170794f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:48.299099 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40e8ae-2c18-4513-8dad-7aceba49ddc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:48.299744 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40e8ae-2c18-4513-8dad-7aceba49ddc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:48.300483 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40e8ae-2c18-4513-8dad-7aceba49ddc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:48.300768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40e8ae-2c18-4513-8dad-7aceba49ddc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:48.301474 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40e8ae-2c18-4513-8dad-7aceba49ddc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:48.303393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40e8ae-2c18-4513-8dad-7aceba49ddc3 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:48.317172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 981/1962] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:48.321416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24580511-1a60-4725-b8fc-f7d2ff4884e5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:48.332313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24580511-1a60-4725-b8fc-f7d2ff4884e5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:48.338156 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0674c2-6b31-4c8b-816e-7c4b2a227072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:48.338614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0674c2-6b31-4c8b-816e-7c4b2a227072 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:48.338898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0674c2-6b31-4c8b-816e-7c4b2a227072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:48.339174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0674c2-6b31-4c8b-816e-7c4b2a227072 admin admin] PATH_INFO: `/v3/projects/07da4be395314be887fc4a677c8634eb/users/32a395b7c7b442568917cc2555d95b34/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:48.354472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0674c2-6b31-4c8b-816e-7c4b2a227072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07da4be395314be887fc4a677c8634eb, user_id=32a395b7c7b442568917cc2555d95b34, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:48.357645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0674c2-6b31-4c8b-816e-7c4b2a227072 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:48.437665 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 982/1963] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:48 2022] PUT /identity/v3/projects/07da4be395314be887fc4a677c8634eb/users/32a395b7c7b442568917cc2555d95b34/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:48.441969 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40e8ae-2c18-4513-8dad-7aceba49ddc3 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:48.458921 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40e8ae-2c18-4513-8dad-7aceba49ddc3 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:48.528195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a5161-ddb1-4551-b873-040e77caceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:48.530625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5161-ddb1-4551-b873-040e77caceac admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:48.531171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5161-ddb1-4551-b873-040e77caceac admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:48.531571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5161-ddb1-4551-b873-040e77caceac admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:48.532289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a5161-ddb1-4551-b873-040e77caceac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:48.534104 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a5161-ddb1-4551-b873-040e77caceac admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:48.547171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 982/1964] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:48.553711 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0674c2-6b31-4c8b-816e-7c4b2a227072 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:48.562441 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0674c2-6b31-4c8b-816e-7c4b2a227072 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:48.567275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b3896-f0ef-464a-a9db-d37c26f126c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:48.567880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b3896-f0ef-464a-a9db-d37c26f126c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:48.568153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b3896-f0ef-464a-a9db-d37c26f126c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:48.568443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b3896-f0ef-464a-a9db-d37c26f126c7 admin admin] PATH_INFO: `/v3/projects/07da4be395314be887fc4a677c8634eb/users/32a395b7c7b442568917cc2555d95b34/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:48.570396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b3896-f0ef-464a-a9db-d37c26f126c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07da4be395314be887fc4a677c8634eb, user_id=32a395b7c7b442568917cc2555d95b34, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:48.573084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b3896-f0ef-464a-a9db-d37c26f126c7 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:48.585786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 983/1965] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:48 2022] PUT /identity/v3/projects/07da4be395314be887fc4a677c8634eb/users/32a395b7c7b442568917cc2555d95b34/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:48.596779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e811c74-3574-4a4f-a24e-70a73dd22ed8 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:48.597096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e811c74-3574-4a4f-a24e-70a73dd22ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:48.602241 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e811c74-3574-4a4f-a24e-70a73dd22ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:48.613634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:48.632949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-6e811c74-3574-4a4f-a24e-70a73dd22ed8 None None] MFA Rules not processed for user `32a395b7c7b442568917cc2555d95b34`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:48.634330 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6e811c74-3574-4a4f-a24e-70a73dd22ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:48.720424 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 983/1966] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:48.733626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b3896-f0ef-464a-a9db-d37c26f126c7 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:48.737985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b83b3896-f0ef-464a-a9db-d37c26f126c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:48.801447 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b3896-f0ef-464a-a9db-d37c26f126c7 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:48.805255 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800892f-13ce-4238-841f-8b2c35d8fb1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:48.808582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800892f-13ce-4238-841f-8b2c35d8fb1f service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:48.808582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800892f-13ce-4238-841f-8b2c35d8fb1f service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:48.808582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800892f-13ce-4238-841f-8b2c35d8fb1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:48.815986 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800892f-13ce-4238-841f-8b2c35d8fb1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:48.817443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800892f-13ce-4238-841f-8b2c35d8fb1f service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:48.827803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 984/1967] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:49.255443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41495f3-0fa7-46a1-afe2-32333b223565 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:49.255728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41495f3-0fa7-46a1-afe2-32333b223565 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:49.256000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41495f3-0fa7-46a1-afe2-32333b223565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:49.275993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-f41495f3-0fa7-46a1-afe2-32333b223565 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:49.277171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f41495f3-0fa7-46a1-afe2-32333b223565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:49.332701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 984/1968] 10.222.0.20 () {58 vars in 967 bytes} [Wed Oct 19 02:34:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:49.346437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800892f-13ce-4238-841f-8b2c35d8fb1f service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:49.347961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8800892f-13ce-4238-841f-8b2c35d8fb1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:49.448610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800892f-13ce-4238-841f-8b2c35d8fb1f service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:49.456538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccf430-8f6d-4888-bd67-dd7bd94076bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:49.457243 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccf430-8f6d-4888-bd67-dd7bd94076bb service swift] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:49.457628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccf430-8f6d-4888-bd67-dd7bd94076bb service swift] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:49.458026 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccf430-8f6d-4888-bd67-dd7bd94076bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:49.479941 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccf430-8f6d-4888-bd67-dd7bd94076bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:49.481765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccf430-8f6d-4888-bd67-dd7bd94076bb service swift] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:49.491004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 985/1969] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:34:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:34:50.526449 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41495f3-0fa7-46a1-afe2-32333b223565 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:50.599211 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41495f3-0fa7-46a1-afe2-32333b223565 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:50.604416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:50.605081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:50.605357 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:50.605626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] PATH_INFO: `/v3/users/35542fae464945a89b39f57dca4194a9` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:50.607453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35542fae464945a89b39f57dca4194a9)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:50.609562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:50.821916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:50.829033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35542fae464945a89b39f57dca4194a9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:50.829691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35542fae464945a89b39f57dca4194a9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:50.956542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35542fae464945a89b39f57dca4194a9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:50.961922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35542fae464945a89b39f57dca4194a9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:50.965668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35542fae464945a89b39f57dca4194a9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:50.984710 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 985/1970] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:50 2022] DELETE /identity/v3/users/35542fae464945a89b39f57dca4194a9 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:50.989421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92f6f6-cd96-4789-85a5-e0f511fd23cd None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:50.990628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92f6f6-cd96-4789-85a5-e0f511fd23cd None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:50.992270 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92f6f6-cd96-4789-85a5-e0f511fd23cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:51.018300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-cd92f6f6-cd96-4789-85a5-e0f511fd23cd None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:51.019872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cd92f6f6-cd96-4789-85a5-e0f511fd23cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:51.051041 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:51.082182 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 986/1971] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:51.091831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:51.093318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:51.169848 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74dbcc-87e8-4192-9d66-4b2d398402c0 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:51.174601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ef41a-e4da-4d6c-9bcf-cbc3aa9d8fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:51.175661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ef41a-e4da-4d6c-9bcf-cbc3aa9d8fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:51.176077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ef41a-e4da-4d6c-9bcf-cbc3aa9d8fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:51.176377 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ef41a-e4da-4d6c-9bcf-cbc3aa9d8fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:51.186768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ef41a-e4da-4d6c-9bcf-cbc3aa9d8fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:51.188270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ef41a-e4da-4d6c-9bcf-cbc3aa9d8fa0 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:51.208733 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 986/1972] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:51.730213 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92f6f6-cd96-4789-85a5-e0f511fd23cd None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:51.731918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cd92f6f6-cd96-4789-85a5-e0f511fd23cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:51.757393 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92f6f6-cd96-4789-85a5-e0f511fd23cd None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:51.763948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:51.764729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:51.765100 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:51.765440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] PATH_INFO: `/v3/projects/576f1b90a8674ff294e6da2bc303a0d2` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:51.766739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=576f1b90a8674ff294e6da2bc303a0d2)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:51.768839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:51.938885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '576f1b90a8674ff294e6da2bc303a0d2', 'request_id': 'req-aecaf07d-78d4-41a8-9408-c780424cf9cc'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:51.945789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '576f1b90a8674ff294e6da2bc303a0d2', 'request_id': 'req-aecaf07d-78d4-41a8-9408-c780424cf9cc'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:52.060646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '576f1b90a8674ff294e6da2bc303a0d2', 'request_id': 'req-aecaf07d-78d4-41a8-9408-c780424cf9cc'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:52.079696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] The token cache is being invalidate because project 576f1b90a8674ff294e6da2bc303a0d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:52.080421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:52.081302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:52.089132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ef41a-e4da-4d6c-9bcf-cbc3aa9d8fa0 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:52.089764 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 987/1973] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:51 2022] DELETE /identity/v3/projects/576f1b90a8674ff294e6da2bc303a0d2 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:52.116413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ef41a-e4da-4d6c-9bcf-cbc3aa9d8fa0 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:52.200526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:52.201219 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:52.201606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:52.201909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] PATH_INFO: `/v3/users/32a395b7c7b442568917cc2555d95b34` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:52.203258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a395b7c7b442568917cc2555d95b34)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:52.207265 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:52.490830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a395b7c7b442568917cc2555d95b34'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:52.491992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a395b7c7b442568917cc2555d95b34'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:52.556113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a395b7c7b442568917cc2555d95b34'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:52.556740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a395b7c7b442568917cc2555d95b34'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:52.565336 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a395b7c7b442568917cc2555d95b34'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:52.567651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:52.598287 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 987/1974] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:52 2022] DELETE /identity/v3/users/32a395b7c7b442568917cc2555d95b34 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:53.544359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:53.546457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:53.563929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecaf07d-78d4-41a8-9408-c780424cf9cc admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:53.633609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e02d9c-251c-428e-bcd1-94866e97bac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:53.634772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e02d9c-251c-428e-bcd1-94866e97bac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:53.635148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e02d9c-251c-428e-bcd1-94866e97bac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:53.635486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e02d9c-251c-428e-bcd1-94866e97bac2 admin admin] PATH_INFO: `/v3/users/22919b0a4b1e4e09b9796a93539547b6` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:53.639356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e02d9c-251c-428e-bcd1-94866e97bac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22919b0a4b1e4e09b9796a93539547b6)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:53.642075 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:53.643599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:53.654570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e02d9c-251c-428e-bcd1-94866e97bac2 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:53.662686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a11ce-bb8a-4757-a6dd-19c07e512dec admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:53.680793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:53.681749 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:53.682163 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:53.682315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] PATH_INFO: `/v3/projects/07da4be395314be887fc4a677c8634eb` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:53.684265 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=07da4be395314be887fc4a677c8634eb)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:53.689924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:53.971602 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-22e02d9c-251c-428e-bcd1-94866e97bac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22919b0a4b1e4e09b9796a93539547b6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:53.972405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-22e02d9c-251c-428e-bcd1-94866e97bac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22919b0a4b1e4e09b9796a93539547b6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:53.972994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07da4be395314be887fc4a677c8634eb', 'request_id': 'req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:53.983068 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07da4be395314be887fc4a677c8634eb', 'request_id': 'req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:53.983874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-22e02d9c-251c-428e-bcd1-94866e97bac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22919b0a4b1e4e09b9796a93539547b6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:53.988758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:54.048300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-22e02d9c-251c-428e-bcd1-94866e97bac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22919b0a4b1e4e09b9796a93539547b6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:54.048969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-22e02d9c-251c-428e-bcd1-94866e97bac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22919b0a4b1e4e09b9796a93539547b6'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:54.067649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 988/1975] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:34:53 2022] DELETE /identity/v3/users/22919b0a4b1e4e09b9796a93539547b6 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:54.068906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc83f0c-27cd-4ae9-8905-74f6e262958b None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:54.069194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc83f0c-27cd-4ae9-8905-74f6e262958b None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:54.069462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc83f0c-27cd-4ae9-8905-74f6e262958b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:54.088639 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-dbc83f0c-27cd-4ae9-8905-74f6e262958b None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:54.089848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-dbc83f0c-27cd-4ae9-8905-74f6e262958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:54.128292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07da4be395314be887fc4a677c8634eb', 'request_id': 'req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:54.148014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] The token cache is being invalidate because project 07da4be395314be887fc4a677c8634eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:54.148541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:54.149601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-49d0d42f-1d20-4df3-8a65-1faa8e5832ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:54.162753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 988/1976] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:53 2022] DELETE /identity/v3/projects/07da4be395314be887fc4a677c8634eb => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:54.168946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47696eaa-b856-49fd-86e6-c2aba600f48c None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:54.168946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47696eaa-b856-49fd-86e6-c2aba600f48c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:54.168946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47696eaa-b856-49fd-86e6-c2aba600f48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:54.202311 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-47696eaa-b856-49fd-86e6-c2aba600f48c None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:54.205008 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-47696eaa-b856-49fd-86e6-c2aba600f48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:54.208751 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 989/1977] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:54.224064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc83f0c-27cd-4ae9-8905-74f6e262958b None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:54.226092 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-dbc83f0c-27cd-4ae9-8905-74f6e262958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:54.315027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 989/1978] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:54.323865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb9e41-ae8e-4bf0-9389-e8217a97be50 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:54.324935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb9e41-ae8e-4bf0-9389-e8217a97be50 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:54.325162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc83f0c-27cd-4ae9-8905-74f6e262958b None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:54.325299 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb9e41-ae8e-4bf0-9389-e8217a97be50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:54.331146 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855925-eebe-4623-a3ac-c6a998192982 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:54.334873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855925-eebe-4623-a3ac-c6a998192982 service swift] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:54.334873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855925-eebe-4623-a3ac-c6a998192982 service swift] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:54.334873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855925-eebe-4623-a3ac-c6a998192982 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:54.342077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de855925-eebe-4623-a3ac-c6a998192982 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:54.344141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de855925-eebe-4623-a3ac-c6a998192982 service swift] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:54.353818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 990/1979] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:34:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:34:54.363720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-3efb9e41-ae8e-4bf0-9389-e8217a97be50 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:54.363720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3efb9e41-ae8e-4bf0-9389-e8217a97be50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:54.374312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 990/1980] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:54.378507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855925-eebe-4623-a3ac-c6a998192982 service swift] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:54.389997 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855925-eebe-4623-a3ac-c6a998192982 service swift] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:54.396305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee955d2-54e8-4f2e-a394-aba65828f426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:54.397139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee955d2-54e8-4f2e-a394-aba65828f426 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:54.397424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee955d2-54e8-4f2e-a394-aba65828f426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:54.397699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee955d2-54e8-4f2e-a394-aba65828f426 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:54.398429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee955d2-54e8-4f2e-a394-aba65828f426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:34:54.398734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee955d2-54e8-4f2e-a394-aba65828f426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:54.400423 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee955d2-54e8-4f2e-a394-aba65828f426 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:54.418113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 991/1981] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:34:54.421135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb9e41-ae8e-4bf0-9389-e8217a97be50 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:54.430872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb9e41-ae8e-4bf0-9389-e8217a97be50 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:54.435817 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28bc14-d782-4201-bd41-636b1dd0ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:54.436426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28bc14-d782-4201-bd41-636b1dd0ff00 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:54.436697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28bc14-d782-4201-bd41-636b1dd0ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:54.437017 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d28bc14-d782-4201-bd41-636b1dd0ff00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:54.437643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28bc14-d782-4201-bd41-636b1dd0ff00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:54.439618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d28bc14-d782-4201-bd41-636b1dd0ff00 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:54.512815 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee955d2-54e8-4f2e-a394-aba65828f426 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:54.514843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8ee955d2-54e8-4f2e-a394-aba65828f426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:54.544436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee955d2-54e8-4f2e-a394-aba65828f426 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:54.560525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:54.561401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:54.561837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:54.562805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] PATH_INFO: `/v3/projects/25e6066db9924f9a9e39a4c1f0356459` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:54.564404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e6066db9924f9a9e39a4c1f0356459)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:54.566492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:54.587109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 991/1982] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:54 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:54.591997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28bc14-d782-4201-bd41-636b1dd0ff00 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:54.650357 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:54.697897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e6066db9924f9a9e39a4c1f0356459', 'request_id': 'req-6e024d15-9652-44d2-8d1f-9212dd01f525'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:54.705396 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e6066db9924f9a9e39a4c1f0356459', 'request_id': 'req-6e024d15-9652-44d2-8d1f-9212dd01f525'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:54.709450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d28bc14-d782-4201-bd41-636b1dd0ff00 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:54.771454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433e65b-ba19-44b6-9b13-5210ada48d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:54.771996 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433e65b-ba19-44b6-9b13-5210ada48d7e admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:54.772264 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433e65b-ba19-44b6-9b13-5210ada48d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:54.772532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433e65b-ba19-44b6-9b13-5210ada48d7e admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:54.773154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433e65b-ba19-44b6-9b13-5210ada48d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:54.775103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433e65b-ba19-44b6-9b13-5210ada48d7e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:54.960096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e6066db9924f9a9e39a4c1f0356459', 'request_id': 'req-6e024d15-9652-44d2-8d1f-9212dd01f525'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:54.974542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] The token cache is being invalidate because project 25e6066db9924f9a9e39a4c1f0356459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:34:54.975233 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:54.975992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6e024d15-9652-44d2-8d1f-9212dd01f525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:34:54.982427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 992/1983] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:34:54 2022] DELETE /identity/v3/projects/25e6066db9924f9a9e39a4c1f0356459 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:54.985091 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1998eb-1498-459e-9aae-960c7bf04ec3 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:54.985587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1998eb-1498-459e-9aae-960c7bf04ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:54.985935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1998eb-1498-459e-9aae-960c7bf04ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:54.988552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 992/1984] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:54 2022] POST /identity/v3/users => generated 290 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:54.992779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433e65b-ba19-44b6-9b13-5210ada48d7e admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:54.995348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8433e65b-ba19-44b6-9b13-5210ada48d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:55.014028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ad1998eb-1498-459e-9aae-960c7bf04ec3 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:55.015731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ad1998eb-1498-459e-9aae-960c7bf04ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:55.026533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433e65b-ba19-44b6-9b13-5210ada48d7e admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:55.028381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 993/1985] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:55.035481 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a672dc-a1af-4fa1-a729-3625925f81ff None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.036096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a672dc-a1af-4fa1-a729-3625925f81ff None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.036618 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a672dc-a1af-4fa1-a729-3625925f81ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.044391 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85efb0c-cf9b-4d35-8bb3-c8799ab34537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:55.045337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85efb0c-cf9b-4d35-8bb3-c8799ab34537 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.045830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85efb0c-cf9b-4d35-8bb3-c8799ab34537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.046317 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85efb0c-cf9b-4d35-8bb3-c8799ab34537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.047471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85efb0c-cf9b-4d35-8bb3-c8799ab34537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:55.051718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85efb0c-cf9b-4d35-8bb3-c8799ab34537 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:55.071478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:DELETE=7,keystone:SELECT=93 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:55.076857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c0a672dc-a1af-4fa1-a729-3625925f81ff None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:55.079583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c0a672dc-a1af-4fa1-a729-3625925f81ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:55.100929 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 993/1986] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:34:55.102633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85efb0c-cf9b-4d35-8bb3-c8799ab34537 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:55.117785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 994/1987] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:55.119198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a672dc-a1af-4fa1-a729-3625925f81ff None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:55.124484 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85efb0c-cf9b-4d35-8bb3-c8799ab34537 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:55.137869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a672dc-a1af-4fa1-a729-3625925f81ff None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:55.146125 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708b831-aee1-4bf2-999e-4558c5b46c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:55.147209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708b831-aee1-4bf2-999e-4558c5b46c76 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.147698 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708b831-aee1-4bf2-999e-4558c5b46c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.148167 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708b831-aee1-4bf2-999e-4558c5b46c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.149135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708b831-aee1-4bf2-999e-4558c5b46c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:55.154087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708b831-aee1-4bf2-999e-4558c5b46c76 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:55.156445 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b62d8-5aa9-41a0-91ac-7da44081c4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:55.156953 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b62d8-5aa9-41a0-91ac-7da44081c4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.159209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b62d8-5aa9-41a0-91ac-7da44081c4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.159650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b62d8-5aa9-41a0-91ac-7da44081c4e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.160483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b62d8-5aa9-41a0-91ac-7da44081c4e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:34:55.161337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b62d8-5aa9-41a0-91ac-7da44081c4e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:55.164394 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b62d8-5aa9-41a0-91ac-7da44081c4e7 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:55.189015 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 994/1988] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:55.191782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708b831-aee1-4bf2-999e-4558c5b46c76 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:55.194355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 995/1989] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:34:55.198872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b62d8-5aa9-41a0-91ac-7da44081c4e7 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:55.208793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708b831-aee1-4bf2-999e-4558c5b46c76 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:55.214935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf772c4-6c30-4a4c-b8dc-1d19990cb35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:55.216137 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf772c4-6c30-4a4c-b8dc-1d19990cb35b admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.216322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b62d8-5aa9-41a0-91ac-7da44081c4e7 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:55.216427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf772c4-6c30-4a4c-b8dc-1d19990cb35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.216677 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf772c4-6c30-4a4c-b8dc-1d19990cb35b admin admin] PATH_INFO: `/v3/projects/2e98a99d79ef470d940dd16869a9083f/users/0bd6670e1a0d46b088c13e82e31dc6da/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.230356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc0244-c834-4d3d-940e-4605f407345a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:55.233942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0244-c834-4d3d-940e-4605f407345a admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.233942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0244-c834-4d3d-940e-4605f407345a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.233942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc0244-c834-4d3d-940e-4605f407345a admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.233942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc0244-c834-4d3d-940e-4605f407345a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:55.235118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc0244-c834-4d3d-940e-4605f407345a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:55.236381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf772c4-6c30-4a4c-b8dc-1d19990cb35b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e98a99d79ef470d940dd16869a9083f, user_id=0bd6670e1a0d46b088c13e82e31dc6da, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:55.243627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf772c4-6c30-4a4c-b8dc-1d19990cb35b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:55.348405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 995/1990] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:55 2022] PUT /identity/v3/projects/2e98a99d79ef470d940dd16869a9083f/users/0bd6670e1a0d46b088c13e82e31dc6da/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:34:55.350235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 996/1991] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:34:55 2022] POST /identity/v3/projects => generated 369 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:34:55.350926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6144551-ef31-469a-a517-8926f7a81cf9 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.351402 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6144551-ef31-469a-a517-8926f7a81cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.351732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6144551-ef31-469a-a517-8926f7a81cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.353965 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc0244-c834-4d3d-940e-4605f407345a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:55.379573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-f6144551-ef31-469a-a517-8926f7a81cf9 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:55.381334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f6144551-ef31-469a-a517-8926f7a81cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:55.446477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc0244-c834-4d3d-940e-4605f407345a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:55.453193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dcae9-816e-4a07-9f4d-446ebca82a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:55.454428 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dcae9-816e-4a07-9f4d-446ebca82a27 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.455025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dcae9-816e-4a07-9f4d-446ebca82a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.455567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dcae9-816e-4a07-9f4d-446ebca82a27 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.457947 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dcae9-816e-4a07-9f4d-446ebca82a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:55.458650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dcae9-816e-4a07-9f4d-446ebca82a27 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:55.463460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 996/1992] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:55.473935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6144551-ef31-469a-a517-8926f7a81cf9 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:55.476152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f6144551-ef31-469a-a517-8926f7a81cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:55.533335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6144551-ef31-469a-a517-8926f7a81cf9 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:55.537336 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4ed5a-73ff-4994-a145-f6a1a0f1ba4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:55.537992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4ed5a-73ff-4994-a145-f6a1a0f1ba4e service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.538319 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4ed5a-73ff-4994-a145-f6a1a0f1ba4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.538595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4ed5a-73ff-4994-a145-f6a1a0f1ba4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.549118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4ed5a-73ff-4994-a145-f6a1a0f1ba4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:55.550625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4ed5a-73ff-4994-a145-f6a1a0f1ba4e service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:55.564572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 997/1993] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:34:55 2022] POST /identity/v3/users => generated 285 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:34:55.566832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 997/1994] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:34:55.575430 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dcae9-816e-4a07-9f4d-446ebca82a27 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:55.591941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4e918-948f-4897-93b4-6855131a00f6 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.592324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dcae9-816e-4a07-9f4d-446ebca82a27 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:55.592435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4e918-948f-4897-93b4-6855131a00f6 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.592675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4e918-948f-4897-93b4-6855131a00f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.598591 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c8910-fd1a-451e-96d3-85527df1f8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:55.599672 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c8910-fd1a-451e-96d3-85527df1f8be admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.599956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c8910-fd1a-451e-96d3-85527df1f8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.600435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c8910-fd1a-451e-96d3-85527df1f8be admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.601231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c8910-fd1a-451e-96d3-85527df1f8be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:55.602808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c8910-fd1a-451e-96d3-85527df1f8be admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:55.616276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 998/1995] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:55.616446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-3ad4e918-948f-4897-93b4-6855131a00f6 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:55.616584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3ad4e918-948f-4897-93b4-6855131a00f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:55.617583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c8910-fd1a-451e-96d3-85527df1f8be admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:55.628297 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c8910-fd1a-451e-96d3-85527df1f8be admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:55.634203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1c2d7-e3ae-44ff-b60a-65b2ac0db6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:55.634807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c2d7-e3ae-44ff-b60a-65b2ac0db6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.635255 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c2d7-e3ae-44ff-b60a-65b2ac0db6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.635546 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1c2d7-e3ae-44ff-b60a-65b2ac0db6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.637566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1c2d7-e3ae-44ff-b60a-65b2ac0db6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:55.639337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1c2d7-e3ae-44ff-b60a-65b2ac0db6c4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:55.653082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 999/1996] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:34:55.657380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1c2d7-e3ae-44ff-b60a-65b2ac0db6c4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:55.667470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1c2d7-e3ae-44ff-b60a-65b2ac0db6c4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:55.672298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb53e9-d244-4f33-a3f0-0aa14e80a480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:55.672944 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb53e9-d244-4f33-a3f0-0aa14e80a480 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.673220 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb53e9-d244-4f33-a3f0-0aa14e80a480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.673490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb53e9-d244-4f33-a3f0-0aa14e80a480 admin admin] PATH_INFO: `/v3/projects/5b01383c479c4ba08949c7f6b803fa63/users/666d512f23054becad2ba38160876c37/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.677745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 998/1997] 10.222.0.20 () {58 vars in 967 bytes} [Wed Oct 19 02:34:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:34:55.696312 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb53e9-d244-4f33-a3f0-0aa14e80a480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b01383c479c4ba08949c7f6b803fa63, user_id=666d512f23054becad2ba38160876c37, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:55.699392 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4e918-948f-4897-93b4-6855131a00f6 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:55.700241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb53e9-d244-4f33-a3f0-0aa14e80a480 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:55.701348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-3ad4e918-948f-4897-93b4-6855131a00f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:55.807128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4e918-948f-4897-93b4-6855131a00f6 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:55.813409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caf45c-7a12-4709-a1b7-540578798309 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:55.814031 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caf45c-7a12-4709-a1b7-540578798309 service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.814333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caf45c-7a12-4709-a1b7-540578798309 service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.815319 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caf45c-7a12-4709-a1b7-540578798309 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.826107 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caf45c-7a12-4709-a1b7-540578798309 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:55.826107 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caf45c-7a12-4709-a1b7-540578798309 service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:55.829148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1000/1998] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:34:55 2022] PUT /identity/v3/projects/5b01383c479c4ba08949c7f6b803fa63/users/666d512f23054becad2ba38160876c37/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:34:55.835575 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 999/1999] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:34:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 19 02:34:55.840729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f26ef-a9ec-4038-bbf7-db6ef87732f0 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:55.841784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f26ef-a9ec-4038-bbf7-db6ef87732f0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:55.842187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f26ef-a9ec-4038-bbf7-db6ef87732f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:55.888226 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-009f26ef-a9ec-4038-bbf7-db6ef87732f0 None None] MFA Rules not processed for user `666d512f23054becad2ba38160876c37`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:55.889581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-009f26ef-a9ec-4038-bbf7-db6ef87732f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:55.915197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:55.928817 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:55.958065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1001/2000] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:34:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:55.973208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caf45c-7a12-4709-a1b7-540578798309 service swift] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:55.973533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e8caf45c-7a12-4709-a1b7-540578798309 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:56.058321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caf45c-7a12-4709-a1b7-540578798309 service swift] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:56.122428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29284117-7479-482e-8095-723a4f4d6dde service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286cfcbb8334421f87c0d426aa52a30a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:56.123039 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29284117-7479-482e-8095-723a4f4d6dde service glance] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:56.123325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29284117-7479-482e-8095-723a4f4d6dde service glance] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:56.123580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29284117-7479-482e-8095-723a4f4d6dde service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:56.132349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29284117-7479-482e-8095-723a4f4d6dde service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:56.133833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29284117-7479-482e-8095-723a4f4d6dde service glance] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:56.143453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1000/2001] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:34:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:34:56.467157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:34:56.701712 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35650c4-c477-4d74-b926-ea496c104598 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:56.702210 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35650c4-c477-4d74-b926-ea496c104598 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:56.702323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35650c4-c477-4d74-b926-ea496c104598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:56.721567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c35650c4-c477-4d74-b926-ea496c104598 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:56.723086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c35650c4-c477-4d74-b926-ea496c104598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:56.779888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1002/2002] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:34:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:56.792668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29284117-7479-482e-8095-723a4f4d6dde service glance] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:56.860319 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29284117-7479-482e-8095-723a4f4d6dde service glance] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:56.865315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c0e3d-8a55-4f1f-ab7c-f47d523cd191 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:56.865925 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c0e3d-8a55-4f1f-ab7c-f47d523cd191 service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:56.866243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c0e3d-8a55-4f1f-ab7c-f47d523cd191 service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:56.866519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c0e3d-8a55-4f1f-ab7c-f47d523cd191 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:56.879687 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c0e3d-8a55-4f1f-ab7c-f47d523cd191 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:56.879687 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c0e3d-8a55-4f1f-ab7c-f47d523cd191 service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:56.889967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1001/2003] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:34:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:34:57.842626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7ecda-c13d-4662-8b77-5f1b42e0ca59 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:57.843628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7ecda-c13d-4662-8b77-5f1b42e0ca59 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:57.844198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7ecda-c13d-4662-8b77-5f1b42e0ca59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:57.872311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-f4a7ecda-c13d-4662-8b77-5f1b42e0ca59 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:34:57.873451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f4a7ecda-c13d-4662-8b77-5f1b42e0ca59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:34:57.881634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1003/2004] 10.222.0.20 () {58 vars in 967 bytes} [Wed Oct 19 02:34:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:34:57.896387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c0e3d-8a55-4f1f-ab7c-f47d523cd191 service swift] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:34:57.944557 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c0e3d-8a55-4f1f-ab7c-f47d523cd191 service swift] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:34:57.953756 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a749e-712f-426c-b1d2-6a4dc4ef6776 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:34:57.954423 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a749e-712f-426c-b1d2-6a4dc4ef6776 service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:34:57.954707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a749e-712f-426c-b1d2-6a4dc4ef6776 service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:34:57.956236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a749e-712f-426c-b1d2-6a4dc4ef6776 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:34:57.967024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a749e-712f-426c-b1d2-6a4dc4ef6776 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:34:57.969414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a749e-712f-426c-b1d2-6a4dc4ef6776 service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:34:57.978611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1002/2005] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:34:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:35:01.596412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7ecda-c13d-4662-8b77-5f1b42e0ca59 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:01.597807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f4a7ecda-c13d-4662-8b77-5f1b42e0ca59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:01.672761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7ecda-c13d-4662-8b77-5f1b42e0ca59 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:01.677771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:01.678792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:01.679142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:01.679575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] PATH_INFO: `/v3/users/8b0f7380468140229b0bb3adf9e9034c` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:01.681059 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b0f7380468140229b0bb3adf9e9034c)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:01.683113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:01.956799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0f7380468140229b0bb3adf9e9034c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:01.957594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0f7380468140229b0bb3adf9e9034c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:01.972183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0f7380468140229b0bb3adf9e9034c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:02.075832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0f7380468140229b0bb3adf9e9034c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:02.078295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0f7380468140229b0bb3adf9e9034c'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:02.081286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:02.117530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1004/2006] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:35:01 2022] DELETE /identity/v3/users/8b0f7380468140229b0bb3adf9e9034c => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:02.123791 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f921f5-ef8a-4a7c-ad7f-f378ebf31c43 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:02.124258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f921f5-ef8a-4a7c-ad7f-f378ebf31c43 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:02.124424 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f921f5-ef8a-4a7c-ad7f-f378ebf31c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:02.147410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c3f921f5-ef8a-4a7c-ad7f-f378ebf31c43 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:02.148968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c3f921f5-ef8a-4a7c-ad7f-f378ebf31c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:02.223438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1003/2007] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:02.233300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:02.234785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:02.309058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10381eb6-dfc8-4f44-ab29-d223f80df3d8 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:02.313187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319cfdc-8f25-4cc8-8743-a770fccc9502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:02.313869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319cfdc-8f25-4cc8-8743-a770fccc9502 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:02.314188 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319cfdc-8f25-4cc8-8743-a770fccc9502 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:02.314625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319cfdc-8f25-4cc8-8743-a770fccc9502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:02.329419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319cfdc-8f25-4cc8-8743-a770fccc9502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:02.330968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319cfdc-8f25-4cc8-8743-a770fccc9502 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:02.341570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1005/2008] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:35:02.672792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f921f5-ef8a-4a7c-ad7f-f378ebf31c43 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:02.672792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c3f921f5-ef8a-4a7c-ad7f-f378ebf31c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:02.690320 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f921f5-ef8a-4a7c-ad7f-f378ebf31c43 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:02.699317 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:02.700252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:02.700679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:02.701093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] PATH_INFO: `/v3/projects/7daf87a0028b4f58a2c69d933579233b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:02.702566 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7daf87a0028b4f58a2c69d933579233b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:02.705587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:02.899382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:02.912806 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7daf87a0028b4f58a2c69d933579233b', 'request_id': 'req-6f6517cf-238a-451a-9ed9-034d4c9209c4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:02.920106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7daf87a0028b4f58a2c69d933579233b', 'request_id': 'req-6f6517cf-238a-451a-9ed9-034d4c9209c4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:03.065742 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7daf87a0028b4f58a2c69d933579233b', 'request_id': 'req-6f6517cf-238a-451a-9ed9-034d4c9209c4'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:03.091874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] The token cache is being invalidate because project 7daf87a0028b4f58a2c69d933579233b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:35:03.096353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:03.100488 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6f6517cf-238a-451a-9ed9-034d4c9209c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:03.107789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1004/2009] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:35:02 2022] DELETE /identity/v3/projects/7daf87a0028b4f58a2c69d933579233b => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:03.121466 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d772afb-d0ca-4e0f-8561-dd70da5db15c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:03.122010 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d772afb-d0ca-4e0f-8561-dd70da5db15c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:03.122548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d772afb-d0ca-4e0f-8561-dd70da5db15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:03.155670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-6d772afb-d0ca-4e0f-8561-dd70da5db15c None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:03.157961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6d772afb-d0ca-4e0f-8561-dd70da5db15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:03.239134 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1006/2010] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:03.246943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f137-f9bf-44e9-9675-52182eb8cec1 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:03.247445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f137-f9bf-44e9-9675-52182eb8cec1 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:03.247770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1f137-f9bf-44e9-9675-52182eb8cec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:03.271894 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b5a1f137-f9bf-44e9-9675-52182eb8cec1 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:03.273252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b5a1f137-f9bf-44e9-9675-52182eb8cec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:03.285724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1005/2011] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:03.286725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d772afb-d0ca-4e0f-8561-dd70da5db15c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:03.300949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d772afb-d0ca-4e0f-8561-dd70da5db15c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:03.307993 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9665d1b-7f35-41bf-9e8b-ce313e4207b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:03.308981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9665d1b-7f35-41bf-9e8b-ce313e4207b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:03.309337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9665d1b-7f35-41bf-9e8b-ce313e4207b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:03.309684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9665d1b-7f35-41bf-9e8b-ce313e4207b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:03.310447 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9665d1b-7f35-41bf-9e8b-ce313e4207b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:35:03.310821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9665d1b-7f35-41bf-9e8b-ce313e4207b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:03.312692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9665d1b-7f35-41bf-9e8b-ce313e4207b4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:03.330932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1007/2012] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:35:03.333924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1f137-f9bf-44e9-9675-52182eb8cec1 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:03.346569 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1f137-f9bf-44e9-9675-52182eb8cec1 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:03.352977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3e753-7014-42bc-b1df-2392df476305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:03.354197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3e753-7014-42bc-b1df-2392df476305 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:03.355247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3e753-7014-42bc-b1df-2392df476305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:03.355719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3e753-7014-42bc-b1df-2392df476305 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:03.356955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3e753-7014-42bc-b1df-2392df476305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:03.359762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3e753-7014-42bc-b1df-2392df476305 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:03.513403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1006/2013] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:35:03 2022] POST /identity/v3/projects => generated 397 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:35:03.517507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9665d1b-7f35-41bf-9e8b-ce313e4207b4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:03.529014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9665d1b-7f35-41bf-9e8b-ce313e4207b4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:03.596597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b959c-62cb-47d3-b332-1a7e178081f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:03.597124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b959c-62cb-47d3-b332-1a7e178081f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:03.597401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b959c-62cb-47d3-b332-1a7e178081f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:03.597809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b959c-62cb-47d3-b332-1a7e178081f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:03.598502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b959c-62cb-47d3-b332-1a7e178081f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:03.600523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b959c-62cb-47d3-b332-1a7e178081f4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:03.738645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1008/2014] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:35:03 2022] POST /identity/v3/users => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:35:03.742356 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3e753-7014-42bc-b1df-2392df476305 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:03.753357 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3e753-7014-42bc-b1df-2392df476305 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:03.758883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4637f-4f5d-4e27-b5ac-15275258b30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:03.759494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4637f-4f5d-4e27-b5ac-15275258b30a admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:03.759821 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4637f-4f5d-4e27-b5ac-15275258b30a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:03.760083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4637f-4f5d-4e27-b5ac-15275258b30a admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:03.760734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4637f-4f5d-4e27-b5ac-15275258b30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:03.763769 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4637f-4f5d-4e27-b5ac-15275258b30a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:03.778387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1007/2015] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:35:03.782756 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b959c-62cb-47d3-b332-1a7e178081f4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:03.787744 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:03.792881 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b959c-62cb-47d3-b332-1a7e178081f4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:03.798084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227c9c3-daa2-4087-a1ae-830bb9dd1b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:03.798876 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c9c3-daa2-4087-a1ae-830bb9dd1b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:03.799388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c9c3-daa2-4087-a1ae-830bb9dd1b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:03.799588 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227c9c3-daa2-4087-a1ae-830bb9dd1b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:03.800594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227c9c3-daa2-4087-a1ae-830bb9dd1b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:03.802067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227c9c3-daa2-4087-a1ae-830bb9dd1b8c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:03.814827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1009/2016] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:35:03.818928 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4637f-4f5d-4e27-b5ac-15275258b30a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:03.839320 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4637f-4f5d-4e27-b5ac-15275258b30a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:03.844453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500fc67-94ba-4f29-8592-7038e490b311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:03.845126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500fc67-94ba-4f29-8592-7038e490b311 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:03.845399 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500fc67-94ba-4f29-8592-7038e490b311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:03.845671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500fc67-94ba-4f29-8592-7038e490b311 admin admin] PATH_INFO: `/v3/projects/654c3704424741aea79bc4e208ef4b64/users/64294c93bef14346a316690ffc88f30e/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:03.876749 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500fc67-94ba-4f29-8592-7038e490b311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654c3704424741aea79bc4e208ef4b64, user_id=64294c93bef14346a316690ffc88f30e, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:03.877942 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500fc67-94ba-4f29-8592-7038e490b311 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:03.991913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1008/2017] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:35:03 2022] PUT /identity/v3/projects/654c3704424741aea79bc4e208ef4b64/users/64294c93bef14346a316690ffc88f30e/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:04.003214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba057ed-da76-4a10-99d1-28f5ae63bf98 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:04.003495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba057ed-da76-4a10-99d1-28f5ae63bf98 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:04.003766 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba057ed-da76-4a10-99d1-28f5ae63bf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:04.055415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-dba057ed-da76-4a10-99d1-28f5ae63bf98 None None] MFA Rules not processed for user `64294c93bef14346a316690ffc88f30e`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:04.056646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-dba057ed-da76-4a10-99d1-28f5ae63bf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:04.216633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500fc67-94ba-4f29-8592-7038e490b311 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:04.218609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f500fc67-94ba-4f29-8592-7038e490b311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:04.218728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1010/2018] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:04 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:04.234207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba057ed-da76-4a10-99d1-28f5ae63bf98 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:04.236411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-dba057ed-da76-4a10-99d1-28f5ae63bf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:04.305599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500fc67-94ba-4f29-8592-7038e490b311 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:04.312334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba057ed-da76-4a10-99d1-28f5ae63bf98 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:04.313348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d38dc3-0819-4b5d-90be-ed0901d041f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:04.313998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d38dc3-0819-4b5d-90be-ed0901d041f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:04.314429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d38dc3-0819-4b5d-90be-ed0901d041f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:04.314719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d38dc3-0819-4b5d-90be-ed0901d041f8 admin admin] PATH_INFO: `/v3/users/5ef49baecf6541e5b7bc331bb133b34f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:04.315835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d38dc3-0819-4b5d-90be-ed0901d041f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ef49baecf6541e5b7bc331bb133b34f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:04.316777 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e23f27-c4c5-4aa0-8639-9644410d78f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:04.317388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e23f27-c4c5-4aa0-8639-9644410d78f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:04.317663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e23f27-c4c5-4aa0-8639-9644410d78f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:04.317761 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d38dc3-0819-4b5d-90be-ed0901d041f8 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:04.317932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e23f27-c4c5-4aa0-8639-9644410d78f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:04.331209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e23f27-c4c5-4aa0-8639-9644410d78f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:04.334094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e23f27-c4c5-4aa0-8639-9644410d78f4 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:04.347120 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1011/2019] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:35:04 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:35:04.348389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28cbf2-d683-43ba-8c3c-597211da2aa9 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:04.348750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28cbf2-d683-43ba-8c3c-597211da2aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:04.349139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28cbf2-d683-43ba-8c3c-597211da2aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:04.385002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9c28cbf2-d683-43ba-8c3c-597211da2aa9 None None] MFA Rules not processed for user `0bd6670e1a0d46b088c13e82e31dc6da`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:04.386387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9c28cbf2-d683-43ba-8c3c-597211da2aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:04.467468 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1012/2020] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:04.472232 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28cbf2-d683-43ba-8c3c-597211da2aa9 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:04.473699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9c28cbf2-d683-43ba-8c3c-597211da2aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:04.494359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28cbf2-d683-43ba-8c3c-597211da2aa9 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:04.499969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a497b-7e12-48cb-a0dd-a7c6d90c9e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:04.500657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a497b-7e12-48cb-a0dd-a7c6d90c9e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:04.501200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a497b-7e12-48cb-a0dd-a7c6d90c9e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:04.501456 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a497b-7e12-48cb-a0dd-a7c6d90c9e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:04.502286 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a497b-7e12-48cb-a0dd-a7c6d90c9e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:04.508752 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a497b-7e12-48cb-a0dd-a7c6d90c9e64 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:04.563903 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-87d38dc3-0819-4b5d-90be-ed0901d041f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ef49baecf6541e5b7bc331bb133b34f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:04.564611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-87d38dc3-0819-4b5d-90be-ed0901d041f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ef49baecf6541e5b7bc331bb133b34f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:04.645222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-87d38dc3-0819-4b5d-90be-ed0901d041f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ef49baecf6541e5b7bc331bb133b34f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:04.646282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-87d38dc3-0819-4b5d-90be-ed0901d041f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ef49baecf6541e5b7bc331bb133b34f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:04.657935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:04.659311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1013/2021] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:35:04 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:35:04.662152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-87d38dc3-0819-4b5d-90be-ed0901d041f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ef49baecf6541e5b7bc331bb133b34f'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:04.663996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a497b-7e12-48cb-a0dd-a7c6d90c9e64 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:04.665912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b48a497b-7e12-48cb-a0dd-a7c6d90c9e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:04.702818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1009/2022] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:35:04 2022] DELETE /identity/v3/users/5ef49baecf6541e5b7bc331bb133b34f => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:04.710879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae067625-c187-4c51-96a3-162bae38a5df None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:04.712399 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae067625-c187-4c51-96a3-162bae38a5df None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:04.712747 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae067625-c187-4c51-96a3-162bae38a5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:04.736272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-ae067625-c187-4c51-96a3-162bae38a5df None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:04.737754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ae067625-c187-4c51-96a3-162bae38a5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:04.753539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a497b-7e12-48cb-a0dd-a7c6d90c9e64 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:04.809888 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1010/2023] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:04.821430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae067625-c187-4c51-96a3-162bae38a5df None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:04.822931 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ae067625-c187-4c51-96a3-162bae38a5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:04.831839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783346-3db6-4da6-88b7-02519f1ba36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:04.832487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783346-3db6-4da6-88b7-02519f1ba36c admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:04.832769 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783346-3db6-4da6-88b7-02519f1ba36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:04.833082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783346-3db6-4da6-88b7-02519f1ba36c admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:04.833704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b783346-3db6-4da6-88b7-02519f1ba36c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:04.835683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b783346-3db6-4da6-88b7-02519f1ba36c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:04.882722 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae067625-c187-4c51-96a3-162bae38a5df None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:04.886563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdd28e-7152-408e-9389-19e77cfffa73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:04.887179 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdd28e-7152-408e-9389-19e77cfffa73 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:04.887462 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdd28e-7152-408e-9389-19e77cfffa73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:04.887734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdd28e-7152-408e-9389-19e77cfffa73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:04.902896 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfdd28e-7152-408e-9389-19e77cfffa73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:04.904362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfdd28e-7152-408e-9389-19e77cfffa73 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:04.915115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1011/2024] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:35:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:35:04.968081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1014/2025] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:35:04 2022] POST /identity/v3/users => generated 296 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:35:04.972033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdd28e-7152-408e-9389-19e77cfffa73 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:04.982864 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdd28e-7152-408e-9389-19e77cfffa73 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:04.992709 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979adf0-62fc-457f-adbe-f79edeffa9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:04.999537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979adf0-62fc-457f-adbe-f79edeffa9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:04.999537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979adf0-62fc-457f-adbe-f79edeffa9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:04.999537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979adf0-62fc-457f-adbe-f79edeffa9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:04.999537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979adf0-62fc-457f-adbe-f79edeffa9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:04.999537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979adf0-62fc-457f-adbe-f79edeffa9b5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:05.008955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1012/2026] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:35:05.014088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783346-3db6-4da6-88b7-02519f1ba36c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:05.030398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b783346-3db6-4da6-88b7-02519f1ba36c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:05.035267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed6400-8efc-427b-aea2-b0a2e345e7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:05.035941 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed6400-8efc-427b-aea2-b0a2e345e7d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:05.036215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed6400-8efc-427b-aea2-b0a2e345e7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:05.036486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed6400-8efc-427b-aea2-b0a2e345e7d1 admin admin] PATH_INFO: `/v3/projects/110490c08a2447a99f76d49efb277de6/users/eb62cc422387437dbef9bd444192cf06/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:05.058116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed6400-8efc-427b-aea2-b0a2e345e7d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=110490c08a2447a99f76d49efb277de6, user_id=eb62cc422387437dbef9bd444192cf06, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:05.060914 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed6400-8efc-427b-aea2-b0a2e345e7d1 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:05.182109 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1015/2027] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:35:05 2022] PUT /identity/v3/projects/110490c08a2447a99f76d49efb277de6/users/eb62cc422387437dbef9bd444192cf06/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:05.185307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979adf0-62fc-457f-adbe-f79edeffa9b5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:05.195770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979adf0-62fc-457f-adbe-f79edeffa9b5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:05.352286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe1d4c-cead-4289-bfa0-abeb09470d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:05.354547 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe1d4c-cead-4289-bfa0-abeb09470d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:05.355088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe1d4c-cead-4289-bfa0-abeb09470d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:05.356280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe1d4c-cead-4289-bfa0-abeb09470d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:05.357426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe1d4c-cead-4289-bfa0-abeb09470d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:05.359558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe1d4c-cead-4289-bfa0-abeb09470d5b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:05.390710 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1013/2028] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:35:05.394298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed6400-8efc-427b-aea2-b0a2e345e7d1 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:05.413103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed6400-8efc-427b-aea2-b0a2e345e7d1 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:05.429760 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80248da-b6e7-4723-963e-ff894cee57d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:05.430626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80248da-b6e7-4723-963e-ff894cee57d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:05.431003 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80248da-b6e7-4723-963e-ff894cee57d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:05.431386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80248da-b6e7-4723-963e-ff894cee57d3 admin admin] PATH_INFO: `/v3/projects/110490c08a2447a99f76d49efb277de6/users/eb62cc422387437dbef9bd444192cf06/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:05.447889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80248da-b6e7-4723-963e-ff894cee57d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=110490c08a2447a99f76d49efb277de6, user_id=eb62cc422387437dbef9bd444192cf06, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:05.456985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80248da-b6e7-4723-963e-ff894cee57d3 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:05.467898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1016/2029] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:35:05 2022] PUT /identity/v3/projects/110490c08a2447a99f76d49efb277de6/users/eb62cc422387437dbef9bd444192cf06/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:05.481142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe1d4c-cead-4289-bfa0-abeb09470d5b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:05.488156 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2afe1d4c-cead-4289-bfa0-abeb09470d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:05.509773 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe1d4c-cead-4289-bfa0-abeb09470d5b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:05.656720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:05.658554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:05.659527 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:05.659834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] PATH_INFO: `/v3/projects/e8645c4c6d2149fcb30d5c57880edf65` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:05.661306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8645c4c6d2149fcb30d5c57880edf65)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:05.663468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:05.837309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8645c4c6d2149fcb30d5c57880edf65', 'request_id': 'req-d13ba462-1b72-43d1-908a-66669f36d42e'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:05.845706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8645c4c6d2149fcb30d5c57880edf65', 'request_id': 'req-d13ba462-1b72-43d1-908a-66669f36d42e'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:05.979766 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8645c4c6d2149fcb30d5c57880edf65', 'request_id': 'req-d13ba462-1b72-43d1-908a-66669f36d42e'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:05.993621 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] The token cache is being invalidate because project e8645c4c6d2149fcb30d5c57880edf65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:35:05.994123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:05.995515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:05.997830 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1014/2030] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:35:05 2022] DELETE /identity/v3/projects/e8645c4c6d2149fcb30d5c57880edf65 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:06.014217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:07.269670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:12.146686 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4812d0-44a8-4912-a291-f4c37cf712b2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:12.147197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4812d0-44a8-4912-a291-f4c37cf712b2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:12.154517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4812d0-44a8-4912-a291-f4c37cf712b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:12.183861 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-fd4812d0-44a8-4912-a291-f4c37cf712b2 None None] MFA Rules not processed for user `eb62cc422387437dbef9bd444192cf06`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:12.185066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-fd4812d0-44a8-4912-a291-f4c37cf712b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:12.236302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:12.266619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1017/2031] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:12.282932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:12.284495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:12.363595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ba462-1b72-43d1-908a-66669f36d42e admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:12.369443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945ceb6-1749-4451-8ad6-c84484ba24c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:12.374865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ceb6-1749-4451-8ad6-c84484ba24c4 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:12.374865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ceb6-1749-4451-8ad6-c84484ba24c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:12.374865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945ceb6-1749-4451-8ad6-c84484ba24c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:12.374865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5945ceb6-1749-4451-8ad6-c84484ba24c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:12.406783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:12.436256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945ceb6-1749-4451-8ad6-c84484ba24c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:12.438003 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945ceb6-1749-4451-8ad6-c84484ba24c4 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:12.447588 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1015/2032] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:35:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:35:14.440022 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4812d0-44a8-4912-a291-f4c37cf712b2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:14.445517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-fd4812d0-44a8-4912-a291-f4c37cf712b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:14.505725 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4812d0-44a8-4912-a291-f4c37cf712b2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:14.510347 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:14.514142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:14.514142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:14.514142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] PATH_INFO: `/v3/users/64294c93bef14346a316690ffc88f30e` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:14.514142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64294c93bef14346a316690ffc88f30e)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:14.515124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:14.756869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64294c93bef14346a316690ffc88f30e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:14.757457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64294c93bef14346a316690ffc88f30e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:14.765331 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64294c93bef14346a316690ffc88f30e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:14.926754 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64294c93bef14346a316690ffc88f30e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:14.927409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64294c93bef14346a316690ffc88f30e'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:14.945593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1018/2033] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:35:14 2022] DELETE /identity/v3/users/64294c93bef14346a316690ffc88f30e => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:14.950804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd6d9e-5a92-42f0-b555-ddb0b020fc0b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:14.951294 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd6d9e-5a92-42f0-b555-ddb0b020fc0b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:14.951400 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd6d9e-5a92-42f0-b555-ddb0b020fc0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:14.970610 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-d5bd6d9e-5a92-42f0-b555-ddb0b020fc0b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:14.976142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d5bd6d9e-5a92-42f0-b555-ddb0b020fc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:15.031024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1016/2034] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:15.040623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:15.042190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:15.100897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35347421-8139-482e-a5fb-c3a8c327dca3 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:15.104679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7b0ba-2730-49c3-88ca-359ef50297b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:15.105304 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b0ba-2730-49c3-88ca-359ef50297b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:15.105576 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b0ba-2730-49c3-88ca-359ef50297b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:15.105848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b0ba-2730-49c3-88ca-359ef50297b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:15.115779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7b0ba-2730-49c3-88ca-359ef50297b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:15.117328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7b0ba-2730-49c3-88ca-359ef50297b2 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:15.121746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:15.128461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1019/2035] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:35:15.602604 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd6d9e-5a92-42f0-b555-ddb0b020fc0b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:15.604383 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d5bd6d9e-5a92-42f0-b555-ddb0b020fc0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:15.622476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd6d9e-5a92-42f0-b555-ddb0b020fc0b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:15.629483 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:15.630570 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:15.631112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:15.631446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] PATH_INFO: `/v3/projects/654c3704424741aea79bc4e208ef4b64` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:15.633267 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=654c3704424741aea79bc4e208ef4b64)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:15.635400 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:15.789136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654c3704424741aea79bc4e208ef4b64', 'request_id': 'req-27945218-d2dd-4745-9ef7-012d6efd3144'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:15.796466 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654c3704424741aea79bc4e208ef4b64', 'request_id': 'req-27945218-d2dd-4745-9ef7-012d6efd3144'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:15.919711 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654c3704424741aea79bc4e208ef4b64', 'request_id': 'req-27945218-d2dd-4745-9ef7-012d6efd3144'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:15.931464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] The token cache is being invalidate because project 654c3704424741aea79bc4e208ef4b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:35:15.931788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:15.932436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-27945218-d2dd-4745-9ef7-012d6efd3144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:15.934504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1017/2036] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:35:15 2022] DELETE /identity/v3/projects/654c3704424741aea79bc4e208ef4b64 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:15.939890 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70ac9e-78bb-490d-9c17-7927516600f3 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:15.940340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70ac9e-78bb-490d-9c17-7927516600f3 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:15.940534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70ac9e-78bb-490d-9c17-7927516600f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:15.962202 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-0b70ac9e-78bb-490d-9c17-7927516600f3 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:15.963474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0b70ac9e-78bb-490d-9c17-7927516600f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:16.047245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1020/2037] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:16.055455 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1fbc1-1852-4121-8845-972f27967dee None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:16.055961 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1fbc1-1852-4121-8845-972f27967dee None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:16.056066 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1fbc1-1852-4121-8845-972f27967dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:16.077078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-05f1fbc1-1852-4121-8845-972f27967dee None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:16.078376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-05f1fbc1-1852-4121-8845-972f27967dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:16.087756 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1018/2038] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:16.092266 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70ac9e-78bb-490d-9c17-7927516600f3 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:16.102475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b70ac9e-78bb-490d-9c17-7927516600f3 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:16.109550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd07c5-4d2f-4d8e-8d51-600b4184f72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:16.110275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd07c5-4d2f-4d8e-8d51-600b4184f72b admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:16.110507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd07c5-4d2f-4d8e-8d51-600b4184f72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:16.110804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd07c5-4d2f-4d8e-8d51-600b4184f72b admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:16.112021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd07c5-4d2f-4d8e-8d51-600b4184f72b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:35:16.112021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd07c5-4d2f-4d8e-8d51-600b4184f72b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:16.113572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd07c5-4d2f-4d8e-8d51-600b4184f72b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:16.127237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1021/2039] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:35:16.131769 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1fbc1-1852-4121-8845-972f27967dee None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:16.141121 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1fbc1-1852-4121-8845-972f27967dee None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:16.146096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152795e7-2dee-418d-9a65-ef738efb2542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:16.146843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152795e7-2dee-418d-9a65-ef738efb2542 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:16.147140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152795e7-2dee-418d-9a65-ef738efb2542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:16.147410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152795e7-2dee-418d-9a65-ef738efb2542 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:16.148106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152795e7-2dee-418d-9a65-ef738efb2542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:16.150242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152795e7-2dee-418d-9a65-ef738efb2542 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:16.322709 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1019/2040] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:35:16 2022] POST /identity/v3/projects => generated 373 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:35:16.327112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd07c5-4d2f-4d8e-8d51-600b4184f72b admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:16.336888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd07c5-4d2f-4d8e-8d51-600b4184f72b admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:16.398704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5984052-d296-4469-bcee-0cc0dcb9815b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:16.406515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5984052-d296-4469-bcee-0cc0dcb9815b admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:16.406515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5984052-d296-4469-bcee-0cc0dcb9815b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:16.406515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5984052-d296-4469-bcee-0cc0dcb9815b admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:16.406515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5984052-d296-4469-bcee-0cc0dcb9815b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:16.406515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5984052-d296-4469-bcee-0cc0dcb9815b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:16.494645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1022/2041] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:35:16 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:35:16.497416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152795e7-2dee-418d-9a65-ef738efb2542 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:16.506395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152795e7-2dee-418d-9a65-ef738efb2542 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:16.511075 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa232ee5-be37-44d1-9f70-e9993f838a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:16.511689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa232ee5-be37-44d1-9f70-e9993f838a60 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:16.511961 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa232ee5-be37-44d1-9f70-e9993f838a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:16.512224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa232ee5-be37-44d1-9f70-e9993f838a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:16.512858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa232ee5-be37-44d1-9f70-e9993f838a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:16.514516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa232ee5-be37-44d1-9f70-e9993f838a60 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:16.527934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1020/2042] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:35:16.529196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5984052-d296-4469-bcee-0cc0dcb9815b admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:16.539854 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5984052-d296-4469-bcee-0cc0dcb9815b admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:16.546153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ae88e-36f2-4a6e-8865-69aba5073e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:16.546786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ae88e-36f2-4a6e-8865-69aba5073e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:16.547064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ae88e-36f2-4a6e-8865-69aba5073e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:16.547332 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ae88e-36f2-4a6e-8865-69aba5073e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:16.547974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ae88e-36f2-4a6e-8865-69aba5073e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:16.549557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ae88e-36f2-4a6e-8865-69aba5073e8e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:16.560454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1023/2043] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:35:16.564679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa232ee5-be37-44d1-9f70-e9993f838a60 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:16.574653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa232ee5-be37-44d1-9f70-e9993f838a60 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:16.579298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3d8ea-7d9f-431c-959c-6c3090e6512b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:16.579948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3d8ea-7d9f-431c-959c-6c3090e6512b admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:16.580220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3d8ea-7d9f-431c-959c-6c3090e6512b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:16.580491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f3d8ea-7d9f-431c-959c-6c3090e6512b admin admin] PATH_INFO: `/v3/projects/4314958c672547cba046eb6c3af918fe/users/a2e0fd0cc64e4323a021ed6a36943986/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:16.593362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:16.596356 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3d8ea-7d9f-431c-959c-6c3090e6512b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4314958c672547cba046eb6c3af918fe, user_id=a2e0fd0cc64e4323a021ed6a36943986, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:16.599076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f3d8ea-7d9f-431c-959c-6c3090e6512b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:16.715964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1021/2044] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:35:16 2022] PUT /identity/v3/projects/4314958c672547cba046eb6c3af918fe/users/a2e0fd0cc64e4323a021ed6a36943986/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:35:16.721161 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486181d-0502-47da-a7a1-00df21c8ac4c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:16.721447 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486181d-0502-47da-a7a1-00df21c8ac4c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:16.721718 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486181d-0502-47da-a7a1-00df21c8ac4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:16.741404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-a486181d-0502-47da-a7a1-00df21c8ac4c None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:16.742642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a486181d-0502-47da-a7a1-00df21c8ac4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:16.825401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1024/2045] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:16.835294 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3d8ea-7d9f-431c-959c-6c3090e6512b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:16.837103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-36f3d8ea-7d9f-431c-959c-6c3090e6512b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:16.898655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f3d8ea-7d9f-431c-959c-6c3090e6512b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:16.903271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1408f-9ef4-465c-9305-d4c6fb9cf5a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:16.904091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1408f-9ef4-465c-9305-d4c6fb9cf5a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:16.904448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1408f-9ef4-465c-9305-d4c6fb9cf5a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:16.904855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1408f-9ef4-465c-9305-d4c6fb9cf5a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:16.930270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1408f-9ef4-465c-9305-d4c6fb9cf5a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:16.931837 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1408f-9ef4-465c-9305-d4c6fb9cf5a0 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:16.946617 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1022/2046] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:35:22.782627 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db38f1-67db-43b3-a1d7-fcca91fccb40 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:22.783265 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db38f1-67db-43b3-a1d7-fcca91fccb40 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:22.783265 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db38f1-67db-43b3-a1d7-fcca91fccb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:22.794636 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:22.814975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-b6db38f1-67db-43b3-a1d7-fcca91fccb40 None None] MFA Rules not processed for user `a2e0fd0cc64e4323a021ed6a36943986`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:22.817459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b6db38f1-67db-43b3-a1d7-fcca91fccb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:22.892945 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1025/2047] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:22.897502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1408f-9ef4-465c-9305-d4c6fb9cf5a0 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:22.909609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1408f-9ef4-465c-9305-d4c6fb9cf5a0 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:22.913626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335b701-8bba-4934-bf1e-8c73756b185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:22.914285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335b701-8bba-4934-bf1e-8c73756b185c admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:22.914643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335b701-8bba-4934-bf1e-8c73756b185c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:22.914818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335b701-8bba-4934-bf1e-8c73756b185c admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:22.915451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335b701-8bba-4934-bf1e-8c73756b185c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:22.917666 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335b701-8bba-4934-bf1e-8c73756b185c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:22.996692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1023/2048] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:35:22 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:35:23.001337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db38f1-67db-43b3-a1d7-fcca91fccb40 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:23.012802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db38f1-67db-43b3-a1d7-fcca91fccb40 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:23.070303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b3afa-a49d-49b5-b0c0-8778dbf6cbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:23.070921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3afa-a49d-49b5-b0c0-8778dbf6cbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:23.071200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3afa-a49d-49b5-b0c0-8778dbf6cbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:23.071472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b3afa-a49d-49b5-b0c0-8778dbf6cbb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:23.072122 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b3afa-a49d-49b5-b0c0-8778dbf6cbb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:23.074657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b3afa-a49d-49b5-b0c0-8778dbf6cbb4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:23.176461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1026/2049] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:35:22 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:35:23.180913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335b701-8bba-4934-bf1e-8c73756b185c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:23.191571 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335b701-8bba-4934-bf1e-8c73756b185c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:23.196763 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919a06c-6e38-4565-9085-463b964fbd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:23.197374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919a06c-6e38-4565-9085-463b964fbd92 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:23.197650 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919a06c-6e38-4565-9085-463b964fbd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:23.197948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919a06c-6e38-4565-9085-463b964fbd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:23.198637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919a06c-6e38-4565-9085-463b964fbd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:23.203001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919a06c-6e38-4565-9085-463b964fbd92 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:23.211738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1024/2050] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:35:23.216461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b3afa-a49d-49b5-b0c0-8778dbf6cbb4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:23.231858 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b3afa-a49d-49b5-b0c0-8778dbf6cbb4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:23.234889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a30e5-7c20-433c-8b4b-0dbcbca98de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:23.235765 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a30e5-7c20-433c-8b4b-0dbcbca98de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:23.236025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a30e5-7c20-433c-8b4b-0dbcbca98de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:23.236157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a30e5-7c20-433c-8b4b-0dbcbca98de5 admin admin] PATH_INFO: `/v3/projects/c07b4c56c1c34f29b7c34edb664bae27/users/8e7e1f922aaa448098051f7bb8ceaca5/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:23.287419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a30e5-7c20-433c-8b4b-0dbcbca98de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07b4c56c1c34f29b7c34edb664bae27, user_id=8e7e1f922aaa448098051f7bb8ceaca5, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:23.290190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a30e5-7c20-433c-8b4b-0dbcbca98de5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:23.389532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1027/2051] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:35:23 2022] PUT /identity/v3/projects/c07b4c56c1c34f29b7c34edb664bae27/users/8e7e1f922aaa448098051f7bb8ceaca5/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:23.393593 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919a06c-6e38-4565-9085-463b964fbd92 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:23.405149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919a06c-6e38-4565-9085-463b964fbd92 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:23.467711 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaa868-63e2-45a2-a9c7-22bb770190fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:23.468324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa868-63e2-45a2-a9c7-22bb770190fe admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:23.468645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa868-63e2-45a2-a9c7-22bb770190fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:23.468860 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa868-63e2-45a2-a9c7-22bb770190fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:23.469497 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaa868-63e2-45a2-a9c7-22bb770190fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:23.471158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaa868-63e2-45a2-a9c7-22bb770190fe admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:23.482606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1025/2052] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:35:23.486541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a30e5-7c20-433c-8b4b-0dbcbca98de5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:23.497572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a30e5-7c20-433c-8b4b-0dbcbca98de5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:23.503064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295eb5ae-0d6c-4097-b24b-67d1fe6f6865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:23.503679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eb5ae-0d6c-4097-b24b-67d1fe6f6865 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:23.503956 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eb5ae-0d6c-4097-b24b-67d1fe6f6865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:23.504693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295eb5ae-0d6c-4097-b24b-67d1fe6f6865 admin admin] PATH_INFO: `/v3/projects/c07b4c56c1c34f29b7c34edb664bae27/users/8e7e1f922aaa448098051f7bb8ceaca5/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:23.508227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295eb5ae-0d6c-4097-b24b-67d1fe6f6865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07b4c56c1c34f29b7c34edb664bae27, user_id=8e7e1f922aaa448098051f7bb8ceaca5, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:23.512248 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295eb5ae-0d6c-4097-b24b-67d1fe6f6865 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:23.523430 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1028/2053] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:35:23 2022] PUT /identity/v3/projects/c07b4c56c1c34f29b7c34edb664bae27/users/8e7e1f922aaa448098051f7bb8ceaca5/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:24.685484 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaa868-63e2-45a2-a9c7-22bb770190fe admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:24.688090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d7eaa868-63e2-45a2-a9c7-22bb770190fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:24.756876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:24.775250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaa868-63e2-45a2-a9c7-22bb770190fe admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:24.855321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963736ac-ce6b-44e6-a8e9-68c5973f699b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:24.855321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963736ac-ce6b-44e6-a8e9-68c5973f699b service placement] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:24.855321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963736ac-ce6b-44e6-a8e9-68c5973f699b service placement] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:24.855321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963736ac-ce6b-44e6-a8e9-68c5973f699b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:24.855666 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-963736ac-ce6b-44e6-a8e9-68c5973f699b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:24.863737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963736ac-ce6b-44e6-a8e9-68c5973f699b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:24.874866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963736ac-ce6b-44e6-a8e9-68c5973f699b service placement] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:24.888463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1026/2054] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:35:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:35:29.903126 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4759f-c1df-4841-baa0-3c643f06866b None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:29.903685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4759f-c1df-4841-baa0-3c643f06866b None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:29.903873 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4759f-c1df-4841-baa0-3c643f06866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:29.933677 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-beb4759f-c1df-4841-baa0-3c643f06866b None None] MFA Rules not processed for user `8e7e1f922aaa448098051f7bb8ceaca5`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:29.935175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-beb4759f-c1df-4841-baa0-3c643f06866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:29.973562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:30.013285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1029/2055] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:30.029352 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963736ac-ce6b-44e6-a8e9-68c5973f699b service placement] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:30.032929 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-963736ac-ce6b-44e6-a8e9-68c5973f699b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:30.111972 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963736ac-ce6b-44e6-a8e9-68c5973f699b service placement] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:30.117680 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd04555-757a-48b5-89a1-bf08503772bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:30.118485 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd04555-757a-48b5-89a1-bf08503772bc service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:30.118827 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd04555-757a-48b5-89a1-bf08503772bc service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:30.119174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd04555-757a-48b5-89a1-bf08503772bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:30.177859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd04555-757a-48b5-89a1-bf08503772bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:30.179489 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd04555-757a-48b5-89a1-bf08503772bc service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:30.189093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1027/2056] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:35:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:35:33.190775 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f7a0a-6275-4911-8223-92c2a9a09d96 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:33.191214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f7a0a-6275-4911-8223-92c2a9a09d96 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:33.191630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f7a0a-6275-4911-8223-92c2a9a09d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:33.211961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c74f7a0a-6275-4911-8223-92c2a9a09d96 None None] MFA Rules not processed for user `a2e0fd0cc64e4323a021ed6a36943986`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:33.213297 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c74f7a0a-6275-4911-8223-92c2a9a09d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:33.220628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1030/2057] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:36.047704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf205-6e26-4b11-95fe-bb6a062889db None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:36.048298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf205-6e26-4b11-95fe-bb6a062889db None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:36.048298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcf205-6e26-4b11-95fe-bb6a062889db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:36.067546 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-28bcf205-6e26-4b11-95fe-bb6a062889db None None] MFA Rules not processed for user `0bd6670e1a0d46b088c13e82e31dc6da`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:36.068858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-28bcf205-6e26-4b11-95fe-bb6a062889db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:36.090346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:36.135656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1028/2058] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:35:37.452934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f7a0a-6275-4911-8223-92c2a9a09d96 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:37.454860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c74f7a0a-6275-4911-8223-92c2a9a09d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:37.530582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f7a0a-6275-4911-8223-92c2a9a09d96 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:37.535006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:37.535645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:37.535932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:37.536207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] PATH_INFO: `/v3/users/666d512f23054becad2ba38160876c37` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:37.537405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=666d512f23054becad2ba38160876c37)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:37.539284 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:37.871651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '666d512f23054becad2ba38160876c37'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:37.872293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '666d512f23054becad2ba38160876c37'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:37.879679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '666d512f23054becad2ba38160876c37'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:37.977612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '666d512f23054becad2ba38160876c37'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:37.978263 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '666d512f23054becad2ba38160876c37'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:38.003128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1031/2059] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:35:37 2022] DELETE /identity/v3/users/666d512f23054becad2ba38160876c37 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:38.007629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825477c-bbfe-4c48-8d5b-7233389fc8d8 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:38.008104 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825477c-bbfe-4c48-8d5b-7233389fc8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:38.008415 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825477c-bbfe-4c48-8d5b-7233389fc8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:38.029339 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-0825477c-bbfe-4c48-8d5b-7233389fc8d8 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:38.030561 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0825477c-bbfe-4c48-8d5b-7233389fc8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:38.091088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1029/2060] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:35:38.102623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:38.104511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:38.155629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:38.166307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babee97-286d-40c5-be32-09aad6c3c011 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:38.170630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc291a-e0e6-4120-b894-ede4affc43a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:38.171255 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc291a-e0e6-4120-b894-ede4affc43a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:38.171529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc291a-e0e6-4120-b894-ede4affc43a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:38.171794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc291a-e0e6-4120-b894-ede4affc43a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:38.181602 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc291a-e0e6-4120-b894-ede4affc43a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:38.183640 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc291a-e0e6-4120-b894-ede4affc43a5 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:38.193240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1032/2061] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:35:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:35:38.624063 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825477c-bbfe-4c48-8d5b-7233389fc8d8 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:38.625802 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0825477c-bbfe-4c48-8d5b-7233389fc8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:38.640374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825477c-bbfe-4c48-8d5b-7233389fc8d8 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:38.647404 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:38.648041 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:38.648232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:38.648649 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] PATH_INFO: `/v3/projects/5b01383c479c4ba08949c7f6b803fa63` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:38.649767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b01383c479c4ba08949c7f6b803fa63)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:38.651750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:38.797299 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b01383c479c4ba08949c7f6b803fa63', 'request_id': 'req-3a17bec1-84a5-41ad-b94c-066624ce4c69'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:38.803818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b01383c479c4ba08949c7f6b803fa63', 'request_id': 'req-3a17bec1-84a5-41ad-b94c-066624ce4c69'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:38.943926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b01383c479c4ba08949c7f6b803fa63', 'request_id': 'req-3a17bec1-84a5-41ad-b94c-066624ce4c69'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:38.955470 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] The token cache is being invalidate because project 5b01383c479c4ba08949c7f6b803fa63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:35:38.955753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:38.956441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3a17bec1-84a5-41ad-b94c-066624ce4c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:35:38.957912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1030/2062] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:35:38 2022] DELETE /identity/v3/projects/5b01383c479c4ba08949c7f6b803fa63 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:38.963327 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77cbac-0d42-4471-bb66-2443e1787018 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:38.963838 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77cbac-0d42-4471-bb66-2443e1787018 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:38.963932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77cbac-0d42-4471-bb66-2443e1787018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:38.983566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-bf77cbac-0d42-4471-bb66-2443e1787018 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:38.985751 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-bf77cbac-0d42-4471-bb66-2443e1787018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:39.045531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1033/2063] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:39.053525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310379c-9b0a-4f3e-9518-d12e5d1da92a None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:39.053984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310379c-9b0a-4f3e-9518-d12e5d1da92a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:39.054113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c310379c-9b0a-4f3e-9518-d12e5d1da92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:39.076120 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c310379c-9b0a-4f3e-9518-d12e5d1da92a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:39.077338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c310379c-9b0a-4f3e-9518-d12e5d1da92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:39.085279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1031/2064] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:39.089568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77cbac-0d42-4471-bb66-2443e1787018 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:39.099424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77cbac-0d42-4471-bb66-2443e1787018 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:39.106227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11362dd-0b81-44f0-94da-ecf9b9e5791b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:39.106670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11362dd-0b81-44f0-94da-ecf9b9e5791b admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:39.106958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11362dd-0b81-44f0-94da-ecf9b9e5791b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:39.107240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11362dd-0b81-44f0-94da-ecf9b9e5791b admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:39.107902 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11362dd-0b81-44f0-94da-ecf9b9e5791b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:35:39.108209 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11362dd-0b81-44f0-94da-ecf9b9e5791b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:39.109817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11362dd-0b81-44f0-94da-ecf9b9e5791b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:39.122600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1034/2065] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:35:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:35:39.127291 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310379c-9b0a-4f3e-9518-d12e5d1da92a None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:39.137135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c310379c-9b0a-4f3e-9518-d12e5d1da92a None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:39.141912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04131d-77c1-4056-82d5-d958cb9d07b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:39.145480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04131d-77c1-4056-82d5-d958cb9d07b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:39.145480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04131d-77c1-4056-82d5-d958cb9d07b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:39.145480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04131d-77c1-4056-82d5-d958cb9d07b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:39.145480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04131d-77c1-4056-82d5-d958cb9d07b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:39.145864 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04131d-77c1-4056-82d5-d958cb9d07b6 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:39.247415 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1032/2066] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:35:39 2022] POST /identity/v3/projects => generated 367 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:35:39.250808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11362dd-0b81-44f0-94da-ecf9b9e5791b admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:39.261886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11362dd-0b81-44f0-94da-ecf9b9e5791b admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:39.323753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720e9a8-b40d-47e0-b494-b8475627b615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:39.324285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720e9a8-b40d-47e0-b494-b8475627b615 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:39.324544 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720e9a8-b40d-47e0-b494-b8475627b615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:39.324819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720e9a8-b40d-47e0-b494-b8475627b615 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:39.325465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720e9a8-b40d-47e0-b494-b8475627b615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:39.327416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0720e9a8-b40d-47e0-b494-b8475627b615 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:39.447549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1035/2067] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:35:39 2022] POST /identity/v3/users => generated 284 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:35:39.451347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04131d-77c1-4056-82d5-d958cb9d07b6 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:39.463138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04131d-77c1-4056-82d5-d958cb9d07b6 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:39.467779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c7796-66bf-480a-800f-40dd3144c08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:39.484429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c7796-66bf-480a-800f-40dd3144c08f admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:39.484429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c7796-66bf-480a-800f-40dd3144c08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:39.484429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c7796-66bf-480a-800f-40dd3144c08f admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:39.484429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c7796-66bf-480a-800f-40dd3144c08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:39.484429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c7796-66bf-480a-800f-40dd3144c08f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:39.484429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1033/2068] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:35:39.485919 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720e9a8-b40d-47e0-b494-b8475627b615 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:39.498332 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720e9a8-b40d-47e0-b494-b8475627b615 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:39.503575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cde30-2e4d-451d-a52d-2bb4de9932c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:39.504180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cde30-2e4d-451d-a52d-2bb4de9932c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:39.504451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cde30-2e4d-451d-a52d-2bb4de9932c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:39.504716 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cde30-2e4d-451d-a52d-2bb4de9932c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:39.505355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cde30-2e4d-451d-a52d-2bb4de9932c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:39.507017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cde30-2e4d-451d-a52d-2bb4de9932c1 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:39.518108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1036/2069] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:35:39.524780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c7796-66bf-480a-800f-40dd3144c08f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:39.536169 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c7796-66bf-480a-800f-40dd3144c08f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:39.541499 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11885dc-34e0-480e-9c95-22005d976304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:39.542216 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11885dc-34e0-480e-9c95-22005d976304 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:39.542506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11885dc-34e0-480e-9c95-22005d976304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:39.542762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11885dc-34e0-480e-9c95-22005d976304 admin admin] PATH_INFO: `/v3/projects/4817f543b70e42da90f1efeab1fa4cfa/users/3a30a0e056944001917f98c357c34049/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:39.556694 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:39.560348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11885dc-34e0-480e-9c95-22005d976304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4817f543b70e42da90f1efeab1fa4cfa, user_id=3a30a0e056944001917f98c357c34049, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:39.563145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11885dc-34e0-480e-9c95-22005d976304 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:39.924921 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1034/2070] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:35:39 2022] PUT /identity/v3/projects/4817f543b70e42da90f1efeab1fa4cfa/users/3a30a0e056944001917f98c357c34049/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:35:39.935401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457188b0-acfc-4c67-97cb-c187ab447cce None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:39.935683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457188b0-acfc-4c67-97cb-c187ab447cce None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:39.935970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457188b0-acfc-4c67-97cb-c187ab447cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:39.967375 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-457188b0-acfc-4c67-97cb-c187ab447cce None None] MFA Rules not processed for user `3a30a0e056944001917f98c357c34049`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:35:39.968595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-457188b0-acfc-4c67-97cb-c187ab447cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:40.011723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:40.036524 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1037/2071] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:35:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:35:40.048364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11885dc-34e0-480e-9c95-22005d976304 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:40.050071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f11885dc-34e0-480e-9c95-22005d976304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:40.112047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11885dc-34e0-480e-9c95-22005d976304 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:40.116250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebff155-6270-4bfe-ae15-b39530affae8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286cfcbb8334421f87c0d426aa52a30a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:40.116892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebff155-6270-4bfe-ae15-b39530affae8 service glance] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:40.117173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebff155-6270-4bfe-ae15-b39530affae8 service glance] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:40.117443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebff155-6270-4bfe-ae15-b39530affae8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:40.126696 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebff155-6270-4bfe-ae15-b39530affae8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:40.128121 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebff155-6270-4bfe-ae15-b39530affae8 service glance] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:40.137323 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1035/2072] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:35:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:35:50.143311 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=29,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:50.222652 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=8 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:35:59.148624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457188b0-acfc-4c67-97cb-c187ab447cce None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:35:59.150165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-457188b0-acfc-4c67-97cb-c187ab447cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:59.209310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457188b0-acfc-4c67-97cb-c187ab447cce None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:35:59.213427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bd3b1-50a6-4c42-a00f-d614d453ba20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:35:59.214133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bd3b1-50a6-4c42-a00f-d614d453ba20 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:35:59.214511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bd3b1-50a6-4c42-a00f-d614d453ba20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:35:59.214973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bd3b1-50a6-4c42-a00f-d614d453ba20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:35:59.216790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-680bd3b1-50a6-4c42-a00f-d614d453ba20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:35:59.292581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bd3b1-50a6-4c42-a00f-d614d453ba20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:35:59.294276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bd3b1-50a6-4c42-a00f-d614d453ba20 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:35:59.311297 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1038/2073] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:35:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:36:09.300125 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=50 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:36:21.847512 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebff155-6270-4bfe-ae15-b39530affae8 service glance] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:21.914871 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebff155-6270-4bfe-ae15-b39530affae8 service glance] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:21.919457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:21.920019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:21.920317 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:21.920572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] PATH_INFO: `/v3/users/3a30a0e056944001917f98c357c34049` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:21.921681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a30a0e056944001917f98c357c34049)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:21.923668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:22.154808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a30a0e056944001917f98c357c34049'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:36:22.155984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a30a0e056944001917f98c357c34049'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:36:22.244797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a30a0e056944001917f98c357c34049'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:36:22.245413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a30a0e056944001917f98c357c34049'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:36:22.253916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a30a0e056944001917f98c357c34049'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:36:22.273511 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1036/2074] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:36:21 2022] DELETE /identity/v3/users/3a30a0e056944001917f98c357c34049 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:36:22.277898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee117be8-79ae-43bc-bae4-485f496139b2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:22.278969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee117be8-79ae-43bc-bae4-485f496139b2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:22.279333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee117be8-79ae-43bc-bae4-485f496139b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:22.298534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ee117be8-79ae-43bc-bae4-485f496139b2 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:36:22.299734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ee117be8-79ae-43bc-bae4-485f496139b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:22.359702 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1039/2075] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:36:22.369385 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:22.370870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:22.429286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0f39b-d316-4fff-b434-225977a31080 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:22.442476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8768a-3801-4873-aa9b-c30fe6eaf64a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:22.442476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8768a-3801-4873-aa9b-c30fe6eaf64a service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:22.442476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8768a-3801-4873-aa9b-c30fe6eaf64a service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:22.442476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8768a-3801-4873-aa9b-c30fe6eaf64a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:22.444346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8768a-3801-4873-aa9b-c30fe6eaf64a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:22.445783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8768a-3801-4873-aa9b-c30fe6eaf64a service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:22.457411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1037/2076] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:36:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:36:22.894922 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee117be8-79ae-43bc-bae4-485f496139b2 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:22.896409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ee117be8-79ae-43bc-bae4-485f496139b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:22.909584 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee117be8-79ae-43bc-bae4-485f496139b2 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:22.914510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:22.915269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:22.915650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:22.916028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] PATH_INFO: `/v3/projects/4817f543b70e42da90f1efeab1fa4cfa` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:22.917298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4817f543b70e42da90f1efeab1fa4cfa)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:22.919363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:23.040659 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4817f543b70e42da90f1efeab1fa4cfa', 'request_id': 'req-000bb721-c5fa-4330-a12a-8672c231162d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:36:23.047642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4817f543b70e42da90f1efeab1fa4cfa', 'request_id': 'req-000bb721-c5fa-4330-a12a-8672c231162d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:36:23.167538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4817f543b70e42da90f1efeab1fa4cfa', 'request_id': 'req-000bb721-c5fa-4330-a12a-8672c231162d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:36:23.178527 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] The token cache is being invalidate because project 4817f543b70e42da90f1efeab1fa4cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:36:23.178812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:36:23.179424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-000bb721-c5fa-4330-a12a-8672c231162d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:36:23.180939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1040/2077] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:36:22 2022] DELETE /identity/v3/projects/4817f543b70e42da90f1efeab1fa4cfa => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:36:23.186413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6d65e-0c3f-4d4b-a8ab-20613d36941b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:23.194864 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6d65e-0c3f-4d4b-a8ab-20613d36941b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:23.194864 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6d65e-0c3f-4d4b-a8ab-20613d36941b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:23.206763 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-22d6d65e-0c3f-4d4b-a8ab-20613d36941b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:36:23.207948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-22d6d65e-0c3f-4d4b-a8ab-20613d36941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:23.224598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:36:23.265945 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1038/2078] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:36:23.273852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c58a0-482e-484b-aed8-35e090f70f57 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:23.274163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c58a0-482e-484b-aed8-35e090f70f57 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:23.274439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c58a0-482e-484b-aed8-35e090f70f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:23.294120 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-8d6c58a0-482e-484b-aed8-35e090f70f57 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:36:23.295296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8d6c58a0-482e-484b-aed8-35e090f70f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:23.303002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1041/2079] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:36:23.307298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6d65e-0c3f-4d4b-a8ab-20613d36941b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:23.317517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6d65e-0c3f-4d4b-a8ab-20613d36941b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:23.322153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d622d-1965-4897-81c1-94fc4070f0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:23.322746 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d622d-1965-4897-81c1-94fc4070f0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:23.323042 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d622d-1965-4897-81c1-94fc4070f0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:23.323312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d622d-1965-4897-81c1-94fc4070f0e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:23.323953 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d622d-1965-4897-81c1-94fc4070f0e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:36:23.324256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d622d-1965-4897-81c1-94fc4070f0e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:23.325847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d622d-1965-4897-81c1-94fc4070f0e1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:23.337962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1039/2080] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:36:23.341949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c58a0-482e-484b-aed8-35e090f70f57 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:23.351178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c58a0-482e-484b-aed8-35e090f70f57 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:23.356005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68812a-bdb7-4c83-94aa-dc8769a378d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:23.356602 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68812a-bdb7-4c83-94aa-dc8769a378d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:23.356876 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68812a-bdb7-4c83-94aa-dc8769a378d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:23.357141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68812a-bdb7-4c83-94aa-dc8769a378d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:23.358271 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68812a-bdb7-4c83-94aa-dc8769a378d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:23.360172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68812a-bdb7-4c83-94aa-dc8769a378d5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:23.462834 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1042/2081] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:36:23 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:36:23.466901 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d622d-1965-4897-81c1-94fc4070f0e1 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:23.477053 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d622d-1965-4897-81c1-94fc4070f0e1 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:23.534488 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627319f3-9a14-4c35-8c6b-eb5f63ef2480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:23.535023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627319f3-9a14-4c35-8c6b-eb5f63ef2480 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:23.535290 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627319f3-9a14-4c35-8c6b-eb5f63ef2480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:23.535555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627319f3-9a14-4c35-8c6b-eb5f63ef2480 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:23.536189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627319f3-9a14-4c35-8c6b-eb5f63ef2480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:23.538302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627319f3-9a14-4c35-8c6b-eb5f63ef2480 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:23.656574 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1040/2082] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:36:23 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:36:23.660723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68812a-bdb7-4c83-94aa-dc8769a378d5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:23.671939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68812a-bdb7-4c83-94aa-dc8769a378d5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:23.676912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610ddef-5a57-4bd7-9a3b-ad75945aef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:23.677518 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610ddef-5a57-4bd7-9a3b-ad75945aef39 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:23.677797 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610ddef-5a57-4bd7-9a3b-ad75945aef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:23.680340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610ddef-5a57-4bd7-9a3b-ad75945aef39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:23.680340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610ddef-5a57-4bd7-9a3b-ad75945aef39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:23.680642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610ddef-5a57-4bd7-9a3b-ad75945aef39 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:23.691834 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1043/2083] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:36:23.695728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627319f3-9a14-4c35-8c6b-eb5f63ef2480 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:23.705688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627319f3-9a14-4c35-8c6b-eb5f63ef2480 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:23.710432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1064233e-9949-4252-8ba8-6c9d91830431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:23.711017 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064233e-9949-4252-8ba8-6c9d91830431 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:23.711296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064233e-9949-4252-8ba8-6c9d91830431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:23.711566 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1064233e-9949-4252-8ba8-6c9d91830431 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:23.712201 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064233e-9949-4252-8ba8-6c9d91830431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:23.713792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1064233e-9949-4252-8ba8-6c9d91830431 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:23.724488 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1041/2084] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:36:23.728529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610ddef-5a57-4bd7-9a3b-ad75945aef39 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:23.738926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610ddef-5a57-4bd7-9a3b-ad75945aef39 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:23.743878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c551412-e32b-4459-b176-2b1e31c6d1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:23.744517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551412-e32b-4459-b176-2b1e31c6d1ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:23.744794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551412-e32b-4459-b176-2b1e31c6d1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:23.745066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551412-e32b-4459-b176-2b1e31c6d1ed admin admin] PATH_INFO: `/v3/projects/9edbfa3d39554f779a87ac3c5a8b766e/users/662eb396a9864d788ea8693b7a9e30a2/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:23.760228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c551412-e32b-4459-b176-2b1e31c6d1ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=9edbfa3d39554f779a87ac3c5a8b766e, user_id=662eb396a9864d788ea8693b7a9e30a2, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:23.762892 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c551412-e32b-4459-b176-2b1e31c6d1ed admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:23.879613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1044/2085] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:36:23 2022] PUT /identity/v3/projects/9edbfa3d39554f779a87ac3c5a8b766e/users/662eb396a9864d788ea8693b7a9e30a2/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:36:23.884683 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886eab7-5c8c-462a-96fa-198c6f06dd75 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:23.884964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886eab7-5c8c-462a-96fa-198c6f06dd75 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:23.887686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886eab7-5c8c-462a-96fa-198c6f06dd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:23.903722 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-d886eab7-5c8c-462a-96fa-198c6f06dd75 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:36:23.905403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d886eab7-5c8c-462a-96fa-198c6f06dd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:23.961098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1042/2086] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:36:23.971160 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c551412-e32b-4459-b176-2b1e31c6d1ed admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:23.972842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0c551412-e32b-4459-b176-2b1e31c6d1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:23.996700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:36:24.031058 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c551412-e32b-4459-b176-2b1e31c6d1ed admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:24.034751 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf481d2a-f151-4d6a-a8d9-61284d964bbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:24.035355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf481d2a-f151-4d6a-a8d9-61284d964bbf service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:24.035693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf481d2a-f151-4d6a-a8d9-61284d964bbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:24.035918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf481d2a-f151-4d6a-a8d9-61284d964bbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:24.044717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf481d2a-f151-4d6a-a8d9-61284d964bbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:24.046813 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf481d2a-f151-4d6a-a8d9-61284d964bbf service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:24.056176 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1045/2087] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:36:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:36:30.174187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afabda7-d3b7-4137-8716-70431a0df578 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:30.174187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afabda7-d3b7-4137-8716-70431a0df578 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:30.174187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afabda7-d3b7-4137-8716-70431a0df578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:30.210546 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-0afabda7-d3b7-4137-8716-70431a0df578 None None] MFA Rules not processed for user `662eb396a9864d788ea8693b7a9e30a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:36:30.210546 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0afabda7-d3b7-4137-8716-70431a0df578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:30.255076 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:36:30.326467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1043/2088] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:36:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:36:30.331139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf481d2a-f151-4d6a-a8d9-61284d964bbf service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:30.348870 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf481d2a-f151-4d6a-a8d9-61284d964bbf service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:30.359355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f79147-afb9-43ba-bb56-8b5c92f0a924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:30.360214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f79147-afb9-43ba-bb56-8b5c92f0a924 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:30.360677 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f79147-afb9-43ba-bb56-8b5c92f0a924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:30.361133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f79147-afb9-43ba-bb56-8b5c92f0a924 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:30.361784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f79147-afb9-43ba-bb56-8b5c92f0a924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:30.364294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f79147-afb9-43ba-bb56-8b5c92f0a924 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:30.509569 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1046/2089] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:36:30 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:36:30.514909 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afabda7-d3b7-4137-8716-70431a0df578 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:30.527748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afabda7-d3b7-4137-8716-70431a0df578 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:30.599973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60dfea-a550-4a5b-9c29-5831fbb12cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:30.600565 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60dfea-a550-4a5b-9c29-5831fbb12cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:30.603165 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60dfea-a550-4a5b-9c29-5831fbb12cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:30.603165 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60dfea-a550-4a5b-9c29-5831fbb12cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:30.603165 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60dfea-a550-4a5b-9c29-5831fbb12cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:30.603878 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60dfea-a550-4a5b-9c29-5831fbb12cd7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:30.734622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1044/2090] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:36:30 2022] POST /identity/v3/users => generated 298 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:36:30.735867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f79147-afb9-43ba-bb56-8b5c92f0a924 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:30.747521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f79147-afb9-43ba-bb56-8b5c92f0a924 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:30.752451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196ae08-09a2-4407-8153-e1f1e2169d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:30.753071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196ae08-09a2-4407-8153-e1f1e2169d32 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:30.753346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196ae08-09a2-4407-8153-e1f1e2169d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:30.753763 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196ae08-09a2-4407-8153-e1f1e2169d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:30.754739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196ae08-09a2-4407-8153-e1f1e2169d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:30.757133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196ae08-09a2-4407-8153-e1f1e2169d32 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:30.771065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1047/2091] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:36:30.780104 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60dfea-a550-4a5b-9c29-5831fbb12cd7 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:30.796978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60dfea-a550-4a5b-9c29-5831fbb12cd7 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:30.802704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55460e2-9b7b-49a0-b72a-36aedde4ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:30.808196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55460e2-9b7b-49a0-b72a-36aedde4ce7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:30.808196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55460e2-9b7b-49a0-b72a-36aedde4ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:30.808196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55460e2-9b7b-49a0-b72a-36aedde4ce7d admin admin] PATH_INFO: `/v3/projects/e5078b16060b4c4dae04d843e40b4174/users/2d5fe3f6223441d584d87db613dbc725/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:30.821531 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55460e2-9b7b-49a0-b72a-36aedde4ce7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5078b16060b4c4dae04d843e40b4174, user_id=2d5fe3f6223441d584d87db613dbc725, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:30.824538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55460e2-9b7b-49a0-b72a-36aedde4ce7d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:30.887908 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1045/2092] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:36:30 2022] PUT /identity/v3/projects/e5078b16060b4c4dae04d843e40b4174/users/2d5fe3f6223441d584d87db613dbc725/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:36:30.892471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196ae08-09a2-4407-8153-e1f1e2169d32 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:30.902801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196ae08-09a2-4407-8153-e1f1e2169d32 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:30.962545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09615c-9810-4a6d-bf13-2958a46a727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:30.963156 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09615c-9810-4a6d-bf13-2958a46a727b admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:30.963427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09615c-9810-4a6d-bf13-2958a46a727b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:30.963694 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09615c-9810-4a6d-bf13-2958a46a727b admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:30.964365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09615c-9810-4a6d-bf13-2958a46a727b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:30.965945 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09615c-9810-4a6d-bf13-2958a46a727b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:30.977690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1048/2093] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:36:30.981367 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55460e2-9b7b-49a0-b72a-36aedde4ce7d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:30.992509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55460e2-9b7b-49a0-b72a-36aedde4ce7d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:30.997222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7f78e-3140-4f42-bd96-8fc2450bbc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:30.998081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f78e-3140-4f42-bd96-8fc2450bbc3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:30.998530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f78e-3140-4f42-bd96-8fc2450bbc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:30.998713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf7f78e-3140-4f42-bd96-8fc2450bbc3e admin admin] PATH_INFO: `/v3/projects/e5078b16060b4c4dae04d843e40b4174/users/2d5fe3f6223441d584d87db613dbc725/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:31.000874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7f78e-3140-4f42-bd96-8fc2450bbc3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5078b16060b4c4dae04d843e40b4174, user_id=2d5fe3f6223441d584d87db613dbc725, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:31.003605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf7f78e-3140-4f42-bd96-8fc2450bbc3e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:31.020464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1046/2094] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:36:30 2022] PUT /identity/v3/projects/e5078b16060b4c4dae04d843e40b4174/users/2d5fe3f6223441d584d87db613dbc725/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:36:39.094931 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85020888-ac74-41f3-af6c-a18d03b41538 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:39.095901 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85020888-ac74-41f3-af6c-a18d03b41538 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:39.096335 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85020888-ac74-41f3-af6c-a18d03b41538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:39.157756 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-85020888-ac74-41f3-af6c-a18d03b41538 None None] MFA Rules not processed for user `2d5fe3f6223441d584d87db613dbc725`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:36:39.159321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-85020888-ac74-41f3-af6c-a18d03b41538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:39.254483 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1049/2095] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:36:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:36:39.274866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7f78e-3140-4f42-bd96-8fc2450bbc3e admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:39.277248 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cbf7f78e-3140-4f42-bd96-8fc2450bbc3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:39.400818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf7f78e-3140-4f42-bd96-8fc2450bbc3e admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:39.407326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f308be7-de5b-4bd8-a751-fc81026c26cd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286cfcbb8334421f87c0d426aa52a30a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:39.408395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308be7-de5b-4bd8-a751-fc81026c26cd service glance] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:39.408884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308be7-de5b-4bd8-a751-fc81026c26cd service glance] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:39.409592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308be7-de5b-4bd8-a751-fc81026c26cd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:39.453351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:36:39.524058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f308be7-de5b-4bd8-a751-fc81026c26cd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:39.527595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f308be7-de5b-4bd8-a751-fc81026c26cd service glance] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:39.539132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1047/2096] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:36:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:36:39.891582 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3d7f9-99c1-41a4-8d8a-cbbdbfc24502 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:39.891894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3d7f9-99c1-41a4-8d8a-cbbdbfc24502 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:39.892167 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3d7f9-99c1-41a4-8d8a-cbbdbfc24502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:39.911067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2ee3d7f9-99c1-41a4-8d8a-cbbdbfc24502 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:36:39.912234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2ee3d7f9-99c1-41a4-8d8a-cbbdbfc24502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:39.917429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:36:39.965525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1050/2097] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:36:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:36:40.129662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f308be7-de5b-4bd8-a751-fc81026c26cd service glance] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:40.131136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5f308be7-de5b-4bd8-a751-fc81026c26cd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:40.203220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f308be7-de5b-4bd8-a751-fc81026c26cd service glance] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:40.208521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa359cb8-00d2-4692-8a55-0da660eb003b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:40.209217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa359cb8-00d2-4692-8a55-0da660eb003b service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:40.214365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa359cb8-00d2-4692-8a55-0da660eb003b service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:40.214365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa359cb8-00d2-4692-8a55-0da660eb003b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:40.218283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa359cb8-00d2-4692-8a55-0da660eb003b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:40.219872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa359cb8-00d2-4692-8a55-0da660eb003b service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:40.228360 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1048/2098] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:36:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:36:42.501870 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3d7f9-99c1-41a4-8d8a-cbbdbfc24502 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:36:42.599448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3d7f9-99c1-41a4-8d8a-cbbdbfc24502 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:36:42.604981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147a8b0-fa84-47bb-a27a-324465ffa94a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:36:42.604981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147a8b0-fa84-47bb-a27a-324465ffa94a service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:42.605290 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147a8b0-fa84-47bb-a27a-324465ffa94a service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:42.605389 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147a8b0-fa84-47bb-a27a-324465ffa94a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:42.608792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9147a8b0-fa84-47bb-a27a-324465ffa94a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:42.741312 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147a8b0-fa84-47bb-a27a-324465ffa94a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:36:42.744013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147a8b0-fa84-47bb-a27a-324465ffa94a service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:36:42.756895 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1051/2099] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:36:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:36:43.020592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48912da-3f15-475d-b7c8-47259743943a None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:43.021189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48912da-3f15-475d-b7c8-47259743943a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:43.021189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48912da-3f15-475d-b7c8-47259743943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:43.058454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c48912da-3f15-475d-b7c8-47259743943a None None] MFA Rules not processed for user `75e5de4fccd243e580a12e2475162451`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:36:43.059537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c48912da-3f15-475d-b7c8-47259743943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:43.191138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1049/2100] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:36:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:36:52.751412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=70 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:36:53.016871 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b918bf-9f4d-42fb-bb13-ec2bf08cad8a None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:53.017156 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b918bf-9f4d-42fb-bb13-ec2bf08cad8a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:53.017431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b918bf-9f4d-42fb-bb13-ec2bf08cad8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:53.036740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-d3b918bf-9f4d-42fb-bb13-ec2bf08cad8a None None] MFA Rules not processed for user `662eb396a9864d788ea8693b7a9e30a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:36:53.037948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d3b918bf-9f4d-42fb-bb13-ec2bf08cad8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:53.044791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1052/2101] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:36:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:36:53.166896 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=73 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:36:54.636217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398f9d2-2bd5-479b-9c3b-128c07ffde13 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:36:54.636869 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398f9d2-2bd5-479b-9c3b-128c07ffde13 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:36:54.636869 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398f9d2-2bd5-479b-9c3b-128c07ffde13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:36:54.656989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b398f9d2-2bd5-479b-9c3b-128c07ffde13 None None] MFA Rules not processed for user `9f0fad4e9be748c786fe114895d8970c`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:36:54.658238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b398f9d2-2bd5-479b-9c3b-128c07ffde13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:36:54.712985 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1050/2102] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:36:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:37:02.087654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c164e16-0c82-4d0c-bf94-24975ebf0bb6 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:37:02.088603 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c164e16-0c82-4d0c-bf94-24975ebf0bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:37:02.088932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c164e16-0c82-4d0c-bf94-24975ebf0bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:37:02.114366 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-3c164e16-0c82-4d0c-bf94-24975ebf0bb6 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:37:02.116414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3c164e16-0c82-4d0c-bf94-24975ebf0bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:37:02.123799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1053/2103] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:37:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:37:02.139803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398f9d2-2bd5-479b-9c3b-128c07ffde13 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:37:02.153995 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398f9d2-2bd5-479b-9c3b-128c07ffde13 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:37:02.159603 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e7d03-8934-4585-ba52-3d82b246e83b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:37:02.160559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e7d03-8934-4585-ba52-3d82b246e83b service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:37:02.161196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e7d03-8934-4585-ba52-3d82b246e83b service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:37:02.162349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e7d03-8934-4585-ba52-3d82b246e83b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:37:02.175258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e7d03-8934-4585-ba52-3d82b246e83b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:37:02.177675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9e7d03-8934-4585-ba52-3d82b246e83b service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:37:02.189200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1051/2104] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:37:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:37:12.107018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=6 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:37:12.185995 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=31 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:37:15.630848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67062d-bac8-4d17-9842-d428a0d43585 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:37:15.631303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67062d-bac8-4d17-9842-d428a0d43585 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:37:15.631420 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67062d-bac8-4d17-9842-d428a0d43585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:37:15.651281 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-0a67062d-bac8-4d17-9842-d428a0d43585 None None] MFA Rules not processed for user `d25a54e03d9848bb9747f18254dadd0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:37:15.652479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0a67062d-bac8-4d17-9842-d428a0d43585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:37:15.706902 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1054/2105] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:37:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:37:16.162127 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e7d03-8934-4585-ba52-3d82b246e83b service swift] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:37:16.163526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6c9e7d03-8934-4585-ba52-3d82b246e83b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:37:16.175755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9e7d03-8934-4585-ba52-3d82b246e83b service swift] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:37:16.179283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9998b6-89e9-48d7-b261-809e54514052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:37:16.179870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9998b6-89e9-48d7-b261-809e54514052 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:37:16.180143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9998b6-89e9-48d7-b261-809e54514052 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:37:16.180411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9998b6-89e9-48d7-b261-809e54514052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:37:16.182147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9e9998b6-89e9-48d7-b261-809e54514052 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:37:16.258321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9998b6-89e9-48d7-b261-809e54514052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:37:16.258321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9998b6-89e9-48d7-b261-809e54514052 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:37:16.267860 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1052/2106] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:37:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:37:17.465607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67062d-bac8-4d17-9842-d428a0d43585 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:37:17.477153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67062d-bac8-4d17-9842-d428a0d43585 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:37:17.481390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ae756-77f7-476c-a96e-75dfe3eec546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:37:17.481999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ae756-77f7-476c-a96e-75dfe3eec546 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:37:17.482307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ae756-77f7-476c-a96e-75dfe3eec546 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:37:17.482577 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ae756-77f7-476c-a96e-75dfe3eec546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:37:17.484547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a67ae756-77f7-476c-a96e-75dfe3eec546 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:37:17.501526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ae756-77f7-476c-a96e-75dfe3eec546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:37:17.503120 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ae756-77f7-476c-a96e-75dfe3eec546 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:37:17.522439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1055/2107] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:37:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:37:26.262953 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=34 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:37:27.510787 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=31 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:37:32.686210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9998b6-89e9-48d7-b261-809e54514052 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:37:32.694865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9998b6-89e9-48d7-b261-809e54514052 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:37:32.698598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c6466-85e1-4a96-a0d4-4e8b93f24110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:37:32.699231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c6466-85e1-4a96-a0d4-4e8b93f24110 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:37:32.699513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c6466-85e1-4a96-a0d4-4e8b93f24110 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:37:32.699788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c6466-85e1-4a96-a0d4-4e8b93f24110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:37:32.708919 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c6466-85e1-4a96-a0d4-4e8b93f24110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:37:32.710374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c6466-85e1-4a96-a0d4-4e8b93f24110 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:37:32.720440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1053/2108] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:37:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:37:39.583392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ae756-77f7-476c-a96e-75dfe3eec546 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:37:39.585594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a67ae756-77f7-476c-a96e-75dfe3eec546 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:37:39.648038 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ae756-77f7-476c-a96e-75dfe3eec546 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:37:39.652159 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32549e-830e-47ea-ba75-5bbc9f83e86e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:37:39.652822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32549e-830e-47ea-ba75-5bbc9f83e86e service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:37:39.653096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32549e-830e-47ea-ba75-5bbc9f83e86e service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:37:39.653364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32549e-830e-47ea-ba75-5bbc9f83e86e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:37:39.655427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ed32549e-830e-47ea-ba75-5bbc9f83e86e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:37:39.666992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32549e-830e-47ea-ba75-5bbc9f83e86e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:37:39.668452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32549e-830e-47ea-ba75-5bbc9f83e86e service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:37:39.677819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1056/2109] 10.222.0.20 () {60 vars in 1396 bytes} [Wed Oct 19 02:37:39 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:37:42.244271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c6466-85e1-4a96-a0d4-4e8b93f24110 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:37:42.245774 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-737c6466-85e1-4a96-a0d4-4e8b93f24110 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:37:42.261398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c6466-85e1-4a96-a0d4-4e8b93f24110 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:37:42.267457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edf8b8-7b94-494b-9429-de77f6f1aa3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:37:42.268041 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edf8b8-7b94-494b-9429-de77f6f1aa3b service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:37:42.268318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edf8b8-7b94-494b-9429-de77f6f1aa3b service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:37:42.268589 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edf8b8-7b94-494b-9429-de77f6f1aa3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:37:42.270332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c5edf8b8-7b94-494b-9429-de77f6f1aa3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:37:42.280742 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edf8b8-7b94-494b-9429-de77f6f1aa3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:37:42.283039 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edf8b8-7b94-494b-9429-de77f6f1aa3b service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:37:42.292176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1054/2110] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:37:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:37:49.673313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=26 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:37:52.287850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=12 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:37:56.818607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32549e-830e-47ea-ba75-5bbc9f83e86e service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:37:56.830992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32549e-830e-47ea-ba75-5bbc9f83e86e service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:37:56.837050 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9f1f8-a157-4d18-a388-1b4b15932b3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:37:56.838294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9f1f8-a157-4d18-a388-1b4b15932b3a service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:37:56.838581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9f1f8-a157-4d18-a388-1b4b15932b3a service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:37:56.838856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9f1f8-a157-4d18-a388-1b4b15932b3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:37:56.840588 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f1a9f1f8-a157-4d18-a388-1b4b15932b3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:37:56.855401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9f1f8-a157-4d18-a388-1b4b15932b3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:37:56.856979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9f1f8-a157-4d18-a388-1b4b15932b3a service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:37:56.867970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1057/2111] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:37:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:04.208288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edf8b8-7b94-494b-9429-de77f6f1aa3b service nova] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:04.218337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edf8b8-7b94-494b-9429-de77f6f1aa3b service nova] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:04.221906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bf64b-7f3a-42f9-9d9f-9d799244d188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:04.222537 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bf64b-7f3a-42f9-9d9f-9d799244d188 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:04.222815 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bf64b-7f3a-42f9-9d9f-9d799244d188 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:04.223082 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bf64b-7f3a-42f9-9d9f-9d799244d188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:04.224815 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-245bf64b-7f3a-42f9-9d9f-9d799244d188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:04.234681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bf64b-7f3a-42f9-9d9f-9d799244d188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:04.236247 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bf64b-7f3a-42f9-9d9f-9d799244d188 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:04.247313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1055/2112] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:06.862951 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=6 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:07.523571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9f1f8-a157-4d18-a388-1b4b15932b3a service nova] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:07.533194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9f1f8-a157-4d18-a388-1b4b15932b3a service nova] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:07.537063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273b4e0-599d-4d9c-ae3c-c0c4113b8ee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:07.537672 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273b4e0-599d-4d9c-ae3c-c0c4113b8ee8 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:07.537948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273b4e0-599d-4d9c-ae3c-c0c4113b8ee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:07.538265 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273b4e0-599d-4d9c-ae3c-c0c4113b8ee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:07.540132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e273b4e0-599d-4d9c-ae3c-c0c4113b8ee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:07.605238 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273b4e0-599d-4d9c-ae3c-c0c4113b8ee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:07.606851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273b4e0-599d-4d9c-ae3c-c0c4113b8ee8 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:07.617918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1058/2113] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:12.192582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7878e6-dd8c-4b27-9c75-be4bc386a91b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:12.193582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7878e6-dd8c-4b27-9c75-be4bc386a91b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:12.194112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7878e6-dd8c-4b27-9c75-be4bc386a91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:12.218489 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-de7878e6-dd8c-4b27-9c75-be4bc386a91b None None] MFA Rules not processed for user `75e5de4fccd243e580a12e2475162451`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:38:12.219149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-de7878e6-dd8c-4b27-9c75-be4bc386a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:12.233465 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1056/2114] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:38:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:38:12.258814 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273b4e0-599d-4d9c-ae3c-c0c4113b8ee8 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:12.260261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e273b4e0-599d-4d9c-ae3c-c0c4113b8ee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:12.409160 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273b4e0-599d-4d9c-ae3c-c0c4113b8ee8 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:12.416456 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2e6d7-abb8-4b43-abda-96d45573fae6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:12.421292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2e6d7-abb8-4b43-abda-96d45573fae6 service placement] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:12.421292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2e6d7-abb8-4b43-abda-96d45573fae6 service placement] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:12.421292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2e6d7-abb8-4b43-abda-96d45573fae6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:12.421292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-38d2e6d7-abb8-4b43-abda-96d45573fae6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:12.437667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2e6d7-abb8-4b43-abda-96d45573fae6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:12.439289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2e6d7-abb8-4b43-abda-96d45573fae6 service placement] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:12.454965 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1059/2115] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:12.457647 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7878e6-dd8c-4b27-9c75-be4bc386a91b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:12.462117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-de7878e6-dd8c-4b27-9c75-be4bc386a91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:12.489760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7878e6-dd8c-4b27-9c75-be4bc386a91b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:12.502446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:12.503089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:12.504194 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:12.504602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] PATH_INFO: `/v3/users/9f0fad4e9be748c786fe114895d8970c` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:12.506286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f0fad4e9be748c786fe114895d8970c)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:12.508369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:12.616015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2e6d7-abb8-4b43-abda-96d45573fae6 service placement] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:12.625660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2e6d7-abb8-4b43-abda-96d45573fae6 service placement] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:12.629249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed893123-f10d-40d0-a0e9-b433533a0418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:12.629840 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed893123-f10d-40d0-a0e9-b433533a0418 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:12.630149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed893123-f10d-40d0-a0e9-b433533a0418 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:12.630425 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed893123-f10d-40d0-a0e9-b433533a0418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:12.632819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ed893123-f10d-40d0-a0e9-b433533a0418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:12.642660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed893123-f10d-40d0-a0e9-b433533a0418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:12.644099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed893123-f10d-40d0-a0e9-b433533a0418 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:12.651974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1060/2116] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:38:12.729850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3bf81-3fee-4031-9139-90c8a2a72736 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:12.730201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3bf81-3fee-4031-9139-90c8a2a72736 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:12.730485 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3bf81-3fee-4031-9139-90c8a2a72736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:12.732938 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f0fad4e9be748c786fe114895d8970c'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:12.733834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f0fad4e9be748c786fe114895d8970c'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:12.751046 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-87f3bf81-3fee-4031-9139-90c8a2a72736 None None] MFA Rules not processed for user `0bd6670e1a0d46b088c13e82e31dc6da`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:38:12.752249 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-87f3bf81-3fee-4031-9139-90c8a2a72736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:12.814917 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f0fad4e9be748c786fe114895d8970c'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:12.815279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1061/2117] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:38:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:38:12.815650 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f0fad4e9be748c786fe114895d8970c'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:12.824075 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f0fad4e9be748c786fe114895d8970c'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:12.844143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1057/2118] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:38:12 2022] DELETE /identity/v3/users/9f0fad4e9be748c786fe114895d8970c => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:38:13.166155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3bf81-3fee-4031-9139-90c8a2a72736 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:13.166282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:13.167315 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-87f3bf81-3fee-4031-9139-90c8a2a72736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:13.167979 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:13.182455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3bf81-3fee-4031-9139-90c8a2a72736 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:13.199632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:13.244472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5f551-3a3d-4e3d-8a8c-cf6242c7323e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:13.245085 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5f551-3a3d-4e3d-8a8c-cf6242c7323e service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:13.245359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5f551-3a3d-4e3d-8a8c-cf6242c7323e service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:13.245760 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5f551-3a3d-4e3d-8a8c-cf6242c7323e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:13.247773 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cbb5f551-3a3d-4e3d-8a8c-cf6242c7323e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:13.252469 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf1de5-96eb-4c2a-998b-2622019cb7b7 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:13.257819 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:13.258824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:13.259282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:13.259747 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] PATH_INFO: `/v3/projects/b01bd7d4ec4d49b180bdd62194c92018` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:13.261427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01bd7d4ec4d49b180bdd62194c92018)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:13.269275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5f551-3a3d-4e3d-8a8c-cf6242c7323e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:13.270161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:13.270269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5f551-3a3d-4e3d-8a8c-cf6242c7323e service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:13.291181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1062/2119] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:13.295759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5f551-3a3d-4e3d-8a8c-cf6242c7323e service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:13.300594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cbb5f551-3a3d-4e3d-8a8c-cf6242c7323e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:13.382871 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01bd7d4ec4d49b180bdd62194c92018', 'request_id': 'req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:13.394433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01bd7d4ec4d49b180bdd62194c92018', 'request_id': 'req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:13.396650 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:13.409923 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5f551-3a3d-4e3d-8a8c-cf6242c7323e service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:13.472735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01bd7d4ec4d49b180bdd62194c92018', 'request_id': 'req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:13.477693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16537c58-997a-4ec5-82a5-421e334e32de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:13.478347 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16537c58-997a-4ec5-82a5-421e334e32de service placement] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:13.481174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16537c58-997a-4ec5-82a5-421e334e32de service placement] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:13.481174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16537c58-997a-4ec5-82a5-421e334e32de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:13.481174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-16537c58-997a-4ec5-82a5-421e334e32de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:13.485109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] The token cache is being invalidate because project b01bd7d4ec4d49b180bdd62194c92018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:38:13.485653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:13.486652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6a0e8664-607e-4156-b7cc-7bcbf6cfd3a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:13.492611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1058/2120] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:38:13 2022] DELETE /identity/v3/projects/b01bd7d4ec4d49b180bdd62194c92018 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:38:13.494445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b53843-ec8e-4ea4-9da4-63c92689a0a4 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:13.494578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16537c58-997a-4ec5-82a5-421e334e32de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:13.495037 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b53843-ec8e-4ea4-9da4-63c92689a0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:13.495490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b53843-ec8e-4ea4-9da4-63c92689a0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:13.496193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16537c58-997a-4ec5-82a5-421e334e32de service placement] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:13.508724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1063/2121] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:13.518944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-99b53843-ec8e-4ea4-9da4-63c92689a0a4 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:38:13.520464 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-99b53843-ec8e-4ea4-9da4-63c92689a0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:13.588695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1059/2122] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:38:13.596566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab9644-c0ee-4c40-bb5a-6390e1eb8ac1 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:13.597140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab9644-c0ee-4c40-bb5a-6390e1eb8ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:13.597140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab9644-c0ee-4c40-bb5a-6390e1eb8ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:13.619547 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-4bab9644-c0ee-4c40-bb5a-6390e1eb8ac1 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:38:13.621059 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4bab9644-c0ee-4c40-bb5a-6390e1eb8ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:13.633492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1064/2123] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:38:13.645354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b53843-ec8e-4ea4-9da4-63c92689a0a4 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:13.659446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b53843-ec8e-4ea4-9da4-63c92689a0a4 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:13.664845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7346383-d553-4e6c-b5be-bb0ec6bf9d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:13.674601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7346383-d553-4e6c-b5be-bb0ec6bf9d13 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:13.674601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7346383-d553-4e6c-b5be-bb0ec6bf9d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:13.674601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7346383-d553-4e6c-b5be-bb0ec6bf9d13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:13.674601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7346383-d553-4e6c-b5be-bb0ec6bf9d13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:38:13.674601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7346383-d553-4e6c-b5be-bb0ec6bf9d13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:13.674601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7346383-d553-4e6c-b5be-bb0ec6bf9d13 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:13.682275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1060/2124] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:38:13.686717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab9644-c0ee-4c40-bb5a-6390e1eb8ac1 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:13.699163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab9644-c0ee-4c40-bb5a-6390e1eb8ac1 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:13.705701 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1fca3-2fec-4d9f-9c6a-734792cc894e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:13.706592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1fca3-2fec-4d9f-9c6a-734792cc894e admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:13.706885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1fca3-2fec-4d9f-9c6a-734792cc894e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:13.712558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1fca3-2fec-4d9f-9c6a-734792cc894e admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:13.712558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1fca3-2fec-4d9f-9c6a-734792cc894e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:13.712558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1fca3-2fec-4d9f-9c6a-734792cc894e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:13.902808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1065/2125] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:38:13 2022] POST /identity/v3/projects => generated 389 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:38:13.906978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7346383-d553-4e6c-b5be-bb0ec6bf9d13 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:13.916717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7346383-d553-4e6c-b5be-bb0ec6bf9d13 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:13.971738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc833948-932c-4592-a870-2bcffa26abd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:13.972374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc833948-932c-4592-a870-2bcffa26abd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:13.972650 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc833948-932c-4592-a870-2bcffa26abd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:13.972919 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc833948-932c-4592-a870-2bcffa26abd8 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:13.973553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc833948-932c-4592-a870-2bcffa26abd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:13.975503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc833948-932c-4592-a870-2bcffa26abd8 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:14.072926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1061/2126] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:38:13 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:38:14.077052 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1fca3-2fec-4d9f-9c6a-734792cc894e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:14.088196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1fca3-2fec-4d9f-9c6a-734792cc894e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:14.092828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e72e9-87dc-4d86-a81b-9165afa9f0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:14.093463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e72e9-87dc-4d86-a81b-9165afa9f0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:14.093748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e72e9-87dc-4d86-a81b-9165afa9f0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:14.094059 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e72e9-87dc-4d86-a81b-9165afa9f0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:14.094705 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e72e9-87dc-4d86-a81b-9165afa9f0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:14.096285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e72e9-87dc-4d86-a81b-9165afa9f0a9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:14.108103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1066/2127] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:38:14.112351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc833948-932c-4592-a870-2bcffa26abd8 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:14.123006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc833948-932c-4592-a870-2bcffa26abd8 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:14.127951 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72b426-ba84-43c2-a4c5-5a725f1fc6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:14.128535 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72b426-ba84-43c2-a4c5-5a725f1fc6ac admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:14.128816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72b426-ba84-43c2-a4c5-5a725f1fc6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:14.129083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72b426-ba84-43c2-a4c5-5a725f1fc6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:14.129719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72b426-ba84-43c2-a4c5-5a725f1fc6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:14.131922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72b426-ba84-43c2-a4c5-5a725f1fc6ac admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:14.142935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1062/2128] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:38:14.147188 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e72e9-87dc-4d86-a81b-9165afa9f0a9 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:14.157188 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e72e9-87dc-4d86-a81b-9165afa9f0a9 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:14.161857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050746c4-3ab2-47ca-8b44-d230fa84be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:14.162528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050746c4-3ab2-47ca-8b44-d230fa84be21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:14.162811 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050746c4-3ab2-47ca-8b44-d230fa84be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:14.163079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050746c4-3ab2-47ca-8b44-d230fa84be21 admin admin] PATH_INFO: `/v3/projects/0d7dfc2343ba4bd3a7f5f39a6765bb36/users/1d16e478c25946a8818641b7d5ea2e15/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:14.171454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:14.179155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050746c4-3ab2-47ca-8b44-d230fa84be21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d7dfc2343ba4bd3a7f5f39a6765bb36, user_id=1d16e478c25946a8818641b7d5ea2e15, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:14.196761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050746c4-3ab2-47ca-8b44-d230fa84be21 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:14.316710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1067/2129] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:38:14 2022] PUT /identity/v3/projects/0d7dfc2343ba4bd3a7f5f39a6765bb36/users/1d16e478c25946a8818641b7d5ea2e15/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:38:14.322524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab659f3-bd6b-4d6e-bccd-c53c483ddf3f None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:14.322912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab659f3-bd6b-4d6e-bccd-c53c483ddf3f None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:14.323141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab659f3-bd6b-4d6e-bccd-c53c483ddf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:14.341714 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-aab659f3-bd6b-4d6e-bccd-c53c483ddf3f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:38:14.342954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-aab659f3-bd6b-4d6e-bccd-c53c483ddf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:14.440178 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:14.458090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1063/2130] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:38:14.465293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050746c4-3ab2-47ca-8b44-d230fa84be21 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:14.467337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-050746c4-3ab2-47ca-8b44-d230fa84be21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:14.530147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050746c4-3ab2-47ca-8b44-d230fa84be21 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:14.533843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf95265-531b-43b1-98da-09cc84f712b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:14.534484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf95265-531b-43b1-98da-09cc84f712b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:14.534758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf95265-531b-43b1-98da-09cc84f712b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:14.535025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf95265-531b-43b1-98da-09cc84f712b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:14.544867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf95265-531b-43b1-98da-09cc84f712b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:14.546311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf95265-531b-43b1-98da-09cc84f712b0 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:14.556473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1068/2131] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:15.031408 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab659f3-bd6b-4d6e-bccd-c53c483ddf3f None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:15.032923 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-aab659f3-bd6b-4d6e-bccd-c53c483ddf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:15.110369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab659f3-bd6b-4d6e-bccd-c53c483ddf3f None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:15.117061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326df2e-9b89-4c13-93d3-ceca84584223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:15.117733 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326df2e-9b89-4c13-93d3-ceca84584223 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:15.118016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326df2e-9b89-4c13-93d3-ceca84584223 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:15.118552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326df2e-9b89-4c13-93d3-ceca84584223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:15.120581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7326df2e-9b89-4c13-93d3-ceca84584223 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:15.199032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326df2e-9b89-4c13-93d3-ceca84584223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:15.200646 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7326df2e-9b89-4c13-93d3-ceca84584223 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:15.211002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1064/2132] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:21.404733 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77781fd-b523-45c7-8288-590bd9f83393 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:21.405170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77781fd-b523-45c7-8288-590bd9f83393 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:21.405373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77781fd-b523-45c7-8288-590bd9f83393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:21.435462 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-d77781fd-b523-45c7-8288-590bd9f83393 None None] MFA Rules not processed for user `1d16e478c25946a8818641b7d5ea2e15`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:38:21.437275 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d77781fd-b523-45c7-8288-590bd9f83393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:21.514856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1069/2133] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:38:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:38:21.529142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326df2e-9b89-4c13-93d3-ceca84584223 service nova] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:21.530680 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7326df2e-9b89-4c13-93d3-ceca84584223 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:21.546497 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326df2e-9b89-4c13-93d3-ceca84584223 service nova] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:21.552355 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dc683-511c-444f-8619-03c2968fb354 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:21.552976 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dc683-511c-444f-8619-03c2968fb354 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:21.553271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dc683-511c-444f-8619-03c2968fb354 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:21.553541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184dc683-511c-444f-8619-03c2968fb354 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:21.562611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184dc683-511c-444f-8619-03c2968fb354 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:21.564526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184dc683-511c-444f-8619-03c2968fb354 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:21.576821 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1065/2134] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:24.862245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961eda-87be-494c-8c42-6892a0e3b8a5 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:24.862771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961eda-87be-494c-8c42-6892a0e3b8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:24.862972 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961eda-87be-494c-8c42-6892a0e3b8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:24.882565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c8961eda-87be-494c-8c42-6892a0e3b8a5 None None] MFA Rules not processed for user `1d16e478c25946a8818641b7d5ea2e15`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:38:24.883804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c8961eda-87be-494c-8c42-6892a0e3b8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:24.891099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1070/2135] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:38:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:38:25.796313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dc683-511c-444f-8619-03c2968fb354 service nova] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:25.819284 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184dc683-511c-444f-8619-03c2968fb354 service nova] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:25.831013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da384a-51c1-4171-9633-12381bb8a7cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:25.832384 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da384a-51c1-4171-9633-12381bb8a7cd service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:25.832835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da384a-51c1-4171-9633-12381bb8a7cd service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:25.833306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da384a-51c1-4171-9633-12381bb8a7cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:25.836754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-61da384a-51c1-4171-9633-12381bb8a7cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:25.858679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da384a-51c1-4171-9633-12381bb8a7cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:25.860814 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da384a-51c1-4171-9633-12381bb8a7cd service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:25.875890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1066/2136] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:34.876894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=66,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:35.870472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=73 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:39.326119 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8961eda-87be-494c-8c42-6892a0e3b8a5 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:39.341530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8961eda-87be-494c-8c42-6892a0e3b8a5 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:39.346228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed73ebe-2c36-4efb-bb48-5945e2751767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:39.346926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed73ebe-2c36-4efb-bb48-5945e2751767 service nova] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:39.347110 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed73ebe-2c36-4efb-bb48-5945e2751767 service nova] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:39.347383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed73ebe-2c36-4efb-bb48-5945e2751767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:39.349806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5ed73ebe-2c36-4efb-bb48-5945e2751767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:39.383912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed73ebe-2c36-4efb-bb48-5945e2751767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:39.385550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed73ebe-2c36-4efb-bb48-5945e2751767 service nova] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:39.397324 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1071/2137] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:49.392263 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=6 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:53.288009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da384a-51c1-4171-9633-12381bb8a7cd service nova] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:53.324770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da384a-51c1-4171-9633-12381bb8a7cd service nova] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:53.328455 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0b65a-7f98-4d83-be03-49de2737d307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:53.329051 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0b65a-7f98-4d83-be03-49de2737d307 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:53.329329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0b65a-7f98-4d83-be03-49de2737d307 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:53.329601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0b65a-7f98-4d83-be03-49de2737d307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:53.331324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ece0b65a-7f98-4d83-be03-49de2737d307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:53.405138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0b65a-7f98-4d83-be03-49de2737d307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:53.406606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0b65a-7f98-4d83-be03-49de2737d307 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:53.425965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1067/2138] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:57.145960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed73ebe-2c36-4efb-bb48-5945e2751767 service nova] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:57.147451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5ed73ebe-2c36-4efb-bb48-5945e2751767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:57.268836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed73ebe-2c36-4efb-bb48-5945e2751767 service nova] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:57.274771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1f394-ce59-49b2-b405-ef2b8bf0268c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:57.275648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1f394-ce59-49b2-b405-ef2b8bf0268c service placement] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:57.275767 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1f394-ce59-49b2-b405-ef2b8bf0268c service placement] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:57.276189 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1f394-ce59-49b2-b405-ef2b8bf0268c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:57.276313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0b65a-7f98-4d83-be03-49de2737d307 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:57.277699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ece0b65a-7f98-4d83-be03-49de2737d307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:57.279517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d8f1f394-ce59-49b2-b405-ef2b8bf0268c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:57.290553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0b65a-7f98-4d83-be03-49de2737d307 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:57.292253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1f394-ce59-49b2-b405-ef2b8bf0268c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:57.293837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1f394-ce59-49b2-b405-ef2b8bf0268c service placement] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:57.295722 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65939fbb-4ecc-4fbf-9aa6-636a79539c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:57.296466 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939fbb-4ecc-4fbf-9aa6-636a79539c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:57.296814 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939fbb-4ecc-4fbf-9aa6-636a79539c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:57.297151 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939fbb-4ecc-4fbf-9aa6-636a79539c80 admin admin] PATH_INFO: `/v3/users/d25a54e03d9848bb9747f18254dadd0b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:57.298424 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65939fbb-4ecc-4fbf-9aa6-636a79539c80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d25a54e03d9848bb9747f18254dadd0b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:57.300566 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65939fbb-4ecc-4fbf-9aa6-636a79539c80 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:57.306904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1072/2139] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:38:57.525524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-65939fbb-4ecc-4fbf-9aa6-636a79539c80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd25a54e03d9848bb9747f18254dadd0b'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:57.526273 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-65939fbb-4ecc-4fbf-9aa6-636a79539c80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd25a54e03d9848bb9747f18254dadd0b'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:57.613681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-65939fbb-4ecc-4fbf-9aa6-636a79539c80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd25a54e03d9848bb9747f18254dadd0b'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:57.614377 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-65939fbb-4ecc-4fbf-9aa6-636a79539c80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd25a54e03d9848bb9747f18254dadd0b'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:57.622254 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-65939fbb-4ecc-4fbf-9aa6-636a79539c80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd25a54e03d9848bb9747f18254dadd0b'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:57.640978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1068/2140] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:38:57 2022] DELETE /identity/v3/users/d25a54e03d9848bb9747f18254dadd0b => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:38:57.940207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1f394-ce59-49b2-b405-ef2b8bf0268c service placement] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:57.952631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1f394-ce59-49b2-b405-ef2b8bf0268c service placement] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:58.011036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:58.011715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:58.011989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:58.012261 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] PATH_INFO: `/v3/projects/96a3bc9cac034979ab4020109cccd48a` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:58.013385 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a3bc9cac034979ab4020109cccd48a)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:58.015298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:58.107382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:58.111162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a3bc9cac034979ab4020109cccd48a', 'request_id': 'req-ed37a4a0-e12c-403e-84c9-1259ca7abea0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:58.117523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a3bc9cac034979ab4020109cccd48a', 'request_id': 'req-ed37a4a0-e12c-403e-84c9-1259ca7abea0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:58.201786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a3bc9cac034979ab4020109cccd48a', 'request_id': 'req-ed37a4a0-e12c-403e-84c9-1259ca7abea0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:58.213323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] The token cache is being invalidate because project 96a3bc9cac034979ab4020109cccd48a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:38:58.213612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:58.214268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ed37a4a0-e12c-403e-84c9-1259ca7abea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:38:58.215808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1073/2141] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:38:57 2022] DELETE /identity/v3/projects/96a3bc9cac034979ab4020109cccd48a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:38:58.220825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa06d1-7b2d-4e4e-b4b3-8ade9234115f None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:58.221110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa06d1-7b2d-4e4e-b4b3-8ade9234115f None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:58.221418 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa06d1-7b2d-4e4e-b4b3-8ade9234115f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:58.271276 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:58.278150 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e2aa06d1-7b2d-4e4e-b4b3-8ade9234115f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:38:58.279347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e2aa06d1-7b2d-4e4e-b4b3-8ade9234115f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:58.337539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1069/2142] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:38:58.346910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab652da9-5eb3-4095-a38c-6b8ace99504b None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:58.347197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab652da9-5eb3-4095-a38c-6b8ace99504b None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:58.347566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab652da9-5eb3-4095-a38c-6b8ace99504b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:58.368033 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ab652da9-5eb3-4095-a38c-6b8ace99504b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:38:58.369139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ab652da9-5eb3-4095-a38c-6b8ace99504b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:58.377248 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1074/2143] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:38:58.382136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa06d1-7b2d-4e4e-b4b3-8ade9234115f None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:58.391738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa06d1-7b2d-4e4e-b4b3-8ade9234115f None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:58.396326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b16ce9-dd94-40ad-8d80-d8e4804b53ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:58.396918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b16ce9-dd94-40ad-8d80-d8e4804b53ff admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:58.397199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b16ce9-dd94-40ad-8d80-d8e4804b53ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:58.397473 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b16ce9-dd94-40ad-8d80-d8e4804b53ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:58.398152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b16ce9-dd94-40ad-8d80-d8e4804b53ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:38:58.398456 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b16ce9-dd94-40ad-8d80-d8e4804b53ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:58.400069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b16ce9-dd94-40ad-8d80-d8e4804b53ff admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:58.411887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1070/2144] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:38:58.416799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab652da9-5eb3-4095-a38c-6b8ace99504b None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:58.425835 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab652da9-5eb3-4095-a38c-6b8ace99504b None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:58.431245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8d699-305e-4848-86ba-4e08899d2d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:58.431955 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d699-305e-4848-86ba-4e08899d2d93 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:58.432326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d699-305e-4848-86ba-4e08899d2d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:58.432601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8d699-305e-4848-86ba-4e08899d2d93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:58.433217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8d699-305e-4848-86ba-4e08899d2d93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:58.435334 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8d699-305e-4848-86ba-4e08899d2d93 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:58.562384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1075/2145] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:38:58 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:38:58.566689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b16ce9-dd94-40ad-8d80-d8e4804b53ff admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:58.577896 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b16ce9-dd94-40ad-8d80-d8e4804b53ff admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:58.633127 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb7a6-e8de-403b-8d57-20a8210f61c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:58.633674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb7a6-e8de-403b-8d57-20a8210f61c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:58.633948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb7a6-e8de-403b-8d57-20a8210f61c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:58.634243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb7a6-e8de-403b-8d57-20a8210f61c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:58.634856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fb7a6-e8de-403b-8d57-20a8210f61c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:58.636779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fb7a6-e8de-403b-8d57-20a8210f61c4 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:58.774659 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1071/2146] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:38:58 2022] POST /identity/v3/users => generated 287 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:38:58.778796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8d699-305e-4848-86ba-4e08899d2d93 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:58.788881 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8d699-305e-4848-86ba-4e08899d2d93 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:58.793345 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfa250-75a3-4a4b-ba1f-daa336bb64e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:58.793935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfa250-75a3-4a4b-ba1f-daa336bb64e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:58.794239 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfa250-75a3-4a4b-ba1f-daa336bb64e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:58.794509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfa250-75a3-4a4b-ba1f-daa336bb64e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:58.795144 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cfa250-75a3-4a4b-ba1f-daa336bb64e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:58.796710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cfa250-75a3-4a4b-ba1f-daa336bb64e8 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:58.806337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1076/2147] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:38:58.815275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb7a6-e8de-403b-8d57-20a8210f61c4 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:58.825671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb7a6-e8de-403b-8d57-20a8210f61c4 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:58.830315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ad42a-023f-4e06-a33f-b58843c332b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:58.830904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ad42a-023f-4e06-a33f-b58843c332b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:58.831183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ad42a-023f-4e06-a33f-b58843c332b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:58.831459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ad42a-023f-4e06-a33f-b58843c332b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:58.832098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ad42a-023f-4e06-a33f-b58843c332b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:58.834364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ad42a-023f-4e06-a33f-b58843c332b7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:58.844805 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1072/2148] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:38:58.849020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfa250-75a3-4a4b-ba1f-daa336bb64e8 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:58.858738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfa250-75a3-4a4b-ba1f-daa336bb64e8 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:58.863197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8352575-6c13-4084-b86f-ea63bdbd35a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:58.863829 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8352575-6c13-4084-b86f-ea63bdbd35a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:58.864110 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8352575-6c13-4084-b86f-ea63bdbd35a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:58.864379 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8352575-6c13-4084-b86f-ea63bdbd35a6 admin admin] PATH_INFO: `/v3/projects/38aeff0edecc43cbb2595dc356238186/users/83277084b1f1462da7b4369191ebb711/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:58.879403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8352575-6c13-4084-b86f-ea63bdbd35a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38aeff0edecc43cbb2595dc356238186, user_id=83277084b1f1462da7b4369191ebb711, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:58.882543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8352575-6c13-4084-b86f-ea63bdbd35a6 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:58.950728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1077/2149] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:38:58 2022] PUT /identity/v3/projects/38aeff0edecc43cbb2595dc356238186/users/83277084b1f1462da7b4369191ebb711/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:38:58.961476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd567743-808e-449c-8d84-98e72169a150 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:58.961754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd567743-808e-449c-8d84-98e72169a150 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:58.962023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd567743-808e-449c-8d84-98e72169a150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:58.990773 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-fd567743-808e-449c-8d84-98e72169a150 None None] MFA Rules not processed for user `83277084b1f1462da7b4369191ebb711`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:38:58.991983 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-fd567743-808e-449c-8d84-98e72169a150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:59.053192 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:59.056739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1073/2150] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:38:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:38:59.061285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8352575-6c13-4084-b86f-ea63bdbd35a6 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:59.124469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8352575-6c13-4084-b86f-ea63bdbd35a6 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:59.129395 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ba1dc-a3a8-4088-8911-aa50cc53efbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:59.130559 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ba1dc-a3a8-4088-8911-aa50cc53efbb admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:59.130836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ba1dc-a3a8-4088-8911-aa50cc53efbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:59.131103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ba1dc-a3a8-4088-8911-aa50cc53efbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:59.131731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ba1dc-a3a8-4088-8911-aa50cc53efbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:59.133612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ba1dc-a3a8-4088-8911-aa50cc53efbb admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:59.214133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1078/2151] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:38:59 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:38:59.218462 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd567743-808e-449c-8d84-98e72169a150 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:59.229117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd567743-808e-449c-8d84-98e72169a150 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:59.284200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d49b3b-5bb9-40df-a53e-fb2aa8a57f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:59.284831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d49b3b-5bb9-40df-a53e-fb2aa8a57f91 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:59.285055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d49b3b-5bb9-40df-a53e-fb2aa8a57f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:59.285321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d49b3b-5bb9-40df-a53e-fb2aa8a57f91 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:59.285937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d49b3b-5bb9-40df-a53e-fb2aa8a57f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:59.287863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d49b3b-5bb9-40df-a53e-fb2aa8a57f91 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:59.378364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1074/2152] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:38:59 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:38:59.385701 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ba1dc-a3a8-4088-8911-aa50cc53efbb admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:59.396281 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ba1dc-a3a8-4088-8911-aa50cc53efbb admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:59.400886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968a0a1-0627-4d05-8a81-4257f6aaa557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:59.401603 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968a0a1-0627-4d05-8a81-4257f6aaa557 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:59.401875 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968a0a1-0627-4d05-8a81-4257f6aaa557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:59.402181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968a0a1-0627-4d05-8a81-4257f6aaa557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:59.407200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968a0a1-0627-4d05-8a81-4257f6aaa557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:59.407200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968a0a1-0627-4d05-8a81-4257f6aaa557 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:59.415266 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1079/2153] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:38:59.420118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d49b3b-5bb9-40df-a53e-fb2aa8a57f91 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:59.432576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d49b3b-5bb9-40df-a53e-fb2aa8a57f91 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:59.437347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b426e-5f2e-4760-876b-d118e18b06eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:59.438001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b426e-5f2e-4760-876b-d118e18b06eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:59.438313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b426e-5f2e-4760-876b-d118e18b06eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:59.438584 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b426e-5f2e-4760-876b-d118e18b06eb admin admin] PATH_INFO: `/v3/projects/3d87ccde5fe2430b8dc92bfeccdabbcf/users/8f66e068b300440eb108c4c151941279/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:59.455603 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b426e-5f2e-4760-876b-d118e18b06eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d87ccde5fe2430b8dc92bfeccdabbcf, user_id=8f66e068b300440eb108c4c151941279, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:59.458348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b426e-5f2e-4760-876b-d118e18b06eb admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:59.532955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1075/2154] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:38:59 2022] PUT /identity/v3/projects/3d87ccde5fe2430b8dc92bfeccdabbcf/users/8f66e068b300440eb108c4c151941279/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:38:59.537723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968a0a1-0627-4d05-8a81-4257f6aaa557 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:59.547841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968a0a1-0627-4d05-8a81-4257f6aaa557 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:59.577928 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:38:59.603183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94bd58-049e-46ff-ac83-5cf32e3ddfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:59.603785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94bd58-049e-46ff-ac83-5cf32e3ddfeb admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:59.604064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94bd58-049e-46ff-ac83-5cf32e3ddfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:59.604335 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94bd58-049e-46ff-ac83-5cf32e3ddfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:59.604961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94bd58-049e-46ff-ac83-5cf32e3ddfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:59.606608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94bd58-049e-46ff-ac83-5cf32e3ddfeb admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:59.616866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1080/2155] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:38:59.621176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b426e-5f2e-4760-876b-d118e18b06eb admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:59.631644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b426e-5f2e-4760-876b-d118e18b06eb admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:59.636421 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b707d-4964-4882-b8a3-9db5b927f369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:59.637062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b707d-4964-4882-b8a3-9db5b927f369 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:59.637335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b707d-4964-4882-b8a3-9db5b927f369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:59.637603 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b707d-4964-4882-b8a3-9db5b927f369 admin admin] PATH_INFO: `/v3/projects/3d87ccde5fe2430b8dc92bfeccdabbcf/users/8f66e068b300440eb108c4c151941279/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:59.639548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b707d-4964-4882-b8a3-9db5b927f369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d87ccde5fe2430b8dc92bfeccdabbcf, user_id=8f66e068b300440eb108c4c151941279, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:59.642223 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b707d-4964-4882-b8a3-9db5b927f369 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:59.652235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1076/2156] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:38:59 2022] PUT /identity/v3/projects/3d87ccde5fe2430b8dc92bfeccdabbcf/users/8f66e068b300440eb108c4c151941279/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:38:59.662821 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824fba6-8d6a-4066-a55c-ff8c014f822f None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:59.663130 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824fba6-8d6a-4066-a55c-ff8c014f822f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:59.663566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824fba6-8d6a-4066-a55c-ff8c014f822f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:59.690635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-b824fba6-8d6a-4066-a55c-ff8c014f822f None None] MFA Rules not processed for user `8f66e068b300440eb108c4c151941279`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:38:59.691837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b824fba6-8d6a-4066-a55c-ff8c014f822f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:59.759192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1081/2157] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:38:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:38:59.770401 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b707d-4964-4882-b8a3-9db5b927f369 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:38:59.772044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2c6b707d-4964-4882-b8a3-9db5b927f369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:38:59.848845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b707d-4964-4882-b8a3-9db5b927f369 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:38:59.852385 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bc8f9-9f48-435c-9996-17120f523429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:38:59.852972 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bc8f9-9f48-435c-9996-17120f523429 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:38:59.853313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bc8f9-9f48-435c-9996-17120f523429 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:38:59.853589 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bc8f9-9f48-435c-9996-17120f523429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:38:59.862576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bc8f9-9f48-435c-9996-17120f523429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:38:59.864047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bc8f9-9f48-435c-9996-17120f523429 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:38:59.873521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1077/2158] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:38:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:39:00.072930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824fba6-8d6a-4066-a55c-ff8c014f822f None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:00.083378 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824fba6-8d6a-4066-a55c-ff8c014f822f None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:00.086989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c991726-6a0f-4a8c-9682-8ba5e03ac627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:00.087639 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c991726-6a0f-4a8c-9682-8ba5e03ac627 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:00.087926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c991726-6a0f-4a8c-9682-8ba5e03ac627 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:00.088198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c991726-6a0f-4a8c-9682-8ba5e03ac627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:00.157181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c991726-6a0f-4a8c-9682-8ba5e03ac627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:00.158674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c991726-6a0f-4a8c-9682-8ba5e03ac627 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:00.168182 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1082/2159] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:39:05.497858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ae1b1-4bbb-4ece-a8b4-631f99d58231 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:05.498956 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ae1b1-4bbb-4ece-a8b4-631f99d58231 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:05.499301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ae1b1-4bbb-4ece-a8b4-631f99d58231 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:05.501291 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1078/2160] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:39:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:39:05.509250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a634fa-d88a-4781-97e9-e95af23004e7 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:05.509846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a634fa-d88a-4781-97e9-e95af23004e7 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:05.509846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a634fa-d88a-4781-97e9-e95af23004e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:05.559203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-39a634fa-d88a-4781-97e9-e95af23004e7 None None] MFA Rules not processed for user `b5ff8583997843258e315413abac172f`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:05.561317 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-39a634fa-d88a-4781-97e9-e95af23004e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:05.595372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:05.617451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1083/2161] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:39:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:39:05.630124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ae1b1-4bbb-4ece-a8b4-631f99d58231 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:05.631962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f18ae1b1-4bbb-4ece-a8b4-631f99d58231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:05.645919 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18ae1b1-4bbb-4ece-a8b4-631f99d58231 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:05.649755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9673d4-8b66-4716-9364-7ee296da8d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:05.650391 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9673d4-8b66-4716-9364-7ee296da8d67 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:05.650675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9673d4-8b66-4716-9364-7ee296da8d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:05.650952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9673d4-8b66-4716-9364-7ee296da8d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:05.658229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9673d4-8b66-4716-9364-7ee296da8d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:05.659654 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9673d4-8b66-4716-9364-7ee296da8d67 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:05.667746 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1079/2162] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:39:05.757207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bca43a-1f15-4198-9c03-278d74f7dc2c None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:05.757749 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bca43a-1f15-4198-9c03-278d74f7dc2c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:05.758031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bca43a-1f15-4198-9c03-278d74f7dc2c None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:05.759497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1084/2163] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:39:05.765106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab35ee-b9fa-46a7-8ac8-c7076212d72f None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:05.765388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab35ee-b9fa-46a7-8ac8-c7076212d72f None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:05.765746 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dab35ee-b9fa-46a7-8ac8-c7076212d72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:05.785863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-9dab35ee-b9fa-46a7-8ac8-c7076212d72f None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:05.787141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9dab35ee-b9fa-46a7-8ac8-c7076212d72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:05.807344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:05.871884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1080/2164] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:39:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:05.883363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bca43a-1f15-4198-9c03-278d74f7dc2c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:05.894624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bca43a-1f15-4198-9c03-278d74f7dc2c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:05.898318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc15ac4-0b14-4971-86aa-86bc59be42be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:05.898991 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc15ac4-0b14-4971-86aa-86bc59be42be service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:05.899341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc15ac4-0b14-4971-86aa-86bc59be42be service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:05.899685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc15ac4-0b14-4971-86aa-86bc59be42be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:05.910217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc15ac4-0b14-4971-86aa-86bc59be42be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:05.911775 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc15ac4-0b14-4971-86aa-86bc59be42be service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:05.922871 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1085/2165] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:39:15.861426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=21 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:15.918028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=14 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:16.442071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cc444-798f-4ef4-b0a0-c02a61d9d914 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:16.442374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cc444-798f-4ef4-b0a0-c02a61d9d914 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:16.444078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cc444-798f-4ef4-b0a0-c02a61d9d914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:16.462479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-bd4cc444-798f-4ef4-b0a0-c02a61d9d914 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:16.463656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-bd4cc444-798f-4ef4-b0a0-c02a61d9d914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:16.522238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1081/2166] 10.222.0.20 () {58 vars in 967 bytes} [Wed Oct 19 02:39:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:16.566650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc15ac4-0b14-4971-86aa-86bc59be42be service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:16.568133 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7cc15ac4-0b14-4971-86aa-86bc59be42be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:16.637762 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc15ac4-0b14-4971-86aa-86bc59be42be service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:16.642808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56060c-8ea9-45a0-b18e-e2b163f4a351 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:16.643409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56060c-8ea9-45a0-b18e-e2b163f4a351 service swift] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:16.643692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56060c-8ea9-45a0-b18e-e2b163f4a351 service swift] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:16.643961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56060c-8ea9-45a0-b18e-e2b163f4a351 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:16.653008 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56060c-8ea9-45a0-b18e-e2b163f4a351 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:16.654706 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56060c-8ea9-45a0-b18e-e2b163f4a351 service swift] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:16.663710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1086/2167] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:39:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:39:26.059712 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cc444-798f-4ef4-b0a0-c02a61d9d914 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:26.061309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-bd4cc444-798f-4ef4-b0a0-c02a61d9d914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:26.140357 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cc444-798f-4ef4-b0a0-c02a61d9d914 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:26.145704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:26.146363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:26.146643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:26.147433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] PATH_INFO: `/v3/users/662eb396a9864d788ea8693b7a9e30a2` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:26.148619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=662eb396a9864d788ea8693b7a9e30a2)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:26.150583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:26.364173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662eb396a9864d788ea8693b7a9e30a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:26.364821 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662eb396a9864d788ea8693b7a9e30a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:26.442008 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662eb396a9864d788ea8693b7a9e30a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:26.442695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662eb396a9864d788ea8693b7a9e30a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:26.452409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662eb396a9864d788ea8693b7a9e30a2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:26.471086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1082/2168] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:39:26 2022] DELETE /identity/v3/users/662eb396a9864d788ea8693b7a9e30a2 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:26.659421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=32 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:26.757647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56060c-8ea9-45a0-b18e-e2b163f4a351 service swift] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:26.768365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56060c-8ea9-45a0-b18e-e2b163f4a351 service swift] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:26.825618 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:26.826268 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:26.826548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:26.826819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] PATH_INFO: `/v3/projects/9edbfa3d39554f779a87ac3c5a8b766e` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:26.827928 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9edbfa3d39554f779a87ac3c5a8b766e)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:26.829856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:26.962860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9edbfa3d39554f779a87ac3c5a8b766e', 'request_id': 'req-c6c57a9e-3b90-4e73-872e-4066e7cd136f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:26.969757 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9edbfa3d39554f779a87ac3c5a8b766e', 'request_id': 'req-c6c57a9e-3b90-4e73-872e-4066e7cd136f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:27.092200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9edbfa3d39554f779a87ac3c5a8b766e', 'request_id': 'req-c6c57a9e-3b90-4e73-872e-4066e7cd136f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:27.105987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] The token cache is being invalidate because project 9edbfa3d39554f779a87ac3c5a8b766e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:39:27.106333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:27.107000 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:27.111537 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1087/2169] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:39:26 2022] DELETE /identity/v3/projects/9edbfa3d39554f779a87ac3c5a8b766e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:39:27.112845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:27.114672 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:27.131191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50975806-a3df-44ec-8ca3-5aba98d5d658 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:27.152364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:27.188115 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:27.188741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:27.189022 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:27.189296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] PATH_INFO: `/v3/users/2d5fe3f6223441d584d87db613dbc725` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:27.190441 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5fe3f6223441d584d87db613dbc725)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:27.192367 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:27.436907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5fe3f6223441d584d87db613dbc725'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:27.437627 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5fe3f6223441d584d87db613dbc725'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:27.532940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5fe3f6223441d584d87db613dbc725'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:27.542617 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5fe3f6223441d584d87db613dbc725'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:27.542617 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5fe3f6223441d584d87db613dbc725'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:27.558630 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1083/2170] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:39:27 2022] DELETE /identity/v3/users/2d5fe3f6223441d584d87db613dbc725 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:27.844422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:27.846087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:27.859731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c57a9e-3b90-4e73-872e-4066e7cd136f admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:27.922543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:27.923131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:27.923406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:27.923675 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] PATH_INFO: `/v3/projects/e5078b16060b4c4dae04d843e40b4174` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:27.924837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5078b16060b4c4dae04d843e40b4174)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:27.926804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:28.030732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:28.043580 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5078b16060b4c4dae04d843e40b4174', 'request_id': 'req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:28.049717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5078b16060b4c4dae04d843e40b4174', 'request_id': 'req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:28.121939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5078b16060b4c4dae04d843e40b4174', 'request_id': 'req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:28.133639 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] The token cache is being invalidate because project e5078b16060b4c4dae04d843e40b4174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:39:28.133923 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:28.134581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:28.136079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1088/2171] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:39:27 2022] DELETE /identity/v3/projects/e5078b16060b4c4dae04d843e40b4174 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:37.664378 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:38.229910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:46.326175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:46.327568 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:46.383171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef88735-fca9-4a06-ac3d-e2cc753e17e9 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:46.430329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb623954-8173-43dc-ac7a-a87eb9da959b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:46.430938 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623954-8173-43dc-ac7a-a87eb9da959b service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:46.431218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623954-8173-43dc-ac7a-a87eb9da959b service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:46.431486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb623954-8173-43dc-ac7a-a87eb9da959b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:46.433245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-bb623954-8173-43dc-ac7a-a87eb9da959b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:46.498806 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb623954-8173-43dc-ac7a-a87eb9da959b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:46.500508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb623954-8173-43dc-ac7a-a87eb9da959b service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:46.511110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1084/2172] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:39:48.646843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:48.649072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:48.664696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c72694-ca6a-4c23-b3a6-5ac0df5c0adb admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:48.669812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:48.670783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:48.671309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:48.671810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] PATH_INFO: `/v3/users/83277084b1f1462da7b4369191ebb711` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:48.673348 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83277084b1f1462da7b4369191ebb711)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:48.675522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:48.913142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83277084b1f1462da7b4369191ebb711'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:48.913808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83277084b1f1462da7b4369191ebb711'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:48.920649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83277084b1f1462da7b4369191ebb711'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:49.006236 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83277084b1f1462da7b4369191ebb711'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:49.006926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83277084b1f1462da7b4369191ebb711'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:49.025385 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1089/2173] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:39:48 2022] DELETE /identity/v3/users/83277084b1f1462da7b4369191ebb711 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:49.029857 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7de2db-d1c3-4185-9674-4aabe86465b7 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:49.030796 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7de2db-d1c3-4185-9674-4aabe86465b7 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:49.031089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7de2db-d1c3-4185-9674-4aabe86465b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:49.049963 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-aa7de2db-d1c3-4185-9674-4aabe86465b7 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:49.051178 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-aa7de2db-d1c3-4185-9674-4aabe86465b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:49.120038 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1085/2174] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:39:49.130094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:49.131543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:49.203812 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27683c5e-424e-4455-b928-d3637a1cb348 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:49.207567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dde38-3d64-4275-bdb9-66d6a9bcfbd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:49.208179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dde38-3d64-4275-bdb9-66d6a9bcfbd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:49.217517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dde38-3d64-4275-bdb9-66d6a9bcfbd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:49.217517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648dde38-3d64-4275-bdb9-66d6a9bcfbd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:49.221861 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dde38-3d64-4275-bdb9-66d6a9bcfbd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:49.223971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648dde38-3d64-4275-bdb9-66d6a9bcfbd7 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:49.233790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1090/2175] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:39:49.797369 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7de2db-d1c3-4185-9674-4aabe86465b7 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:49.799472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-aa7de2db-d1c3-4185-9674-4aabe86465b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:49.826363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7de2db-d1c3-4185-9674-4aabe86465b7 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:49.832654 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:49.833798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:49.834241 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:49.834579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] PATH_INFO: `/v3/projects/38aeff0edecc43cbb2595dc356238186` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:49.836411 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] RBAC: Authorizing `identity:delete_project(project_id=38aeff0edecc43cbb2595dc356238186)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:49.839142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:49.840676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:50.028506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38aeff0edecc43cbb2595dc356238186', 'request_id': 'req-beea5fb0-ffa9-42e0-bb15-8de81e1666af'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.036308 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38aeff0edecc43cbb2595dc356238186', 'request_id': 'req-beea5fb0-ffa9-42e0-bb15-8de81e1666af'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.120026 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38aeff0edecc43cbb2595dc356238186', 'request_id': 'req-beea5fb0-ffa9-42e0-bb15-8de81e1666af'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.131668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] The token cache is being invalidate because project 38aeff0edecc43cbb2595dc356238186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:39:50.131978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.133109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.134634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1086/2176] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:39:49 2022] DELETE /identity/v3/projects/38aeff0edecc43cbb2595dc356238186 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:50.138751 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dde38-3d64-4275-bdb9-66d6a9bcfbd7 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:50.149235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648dde38-3d64-4275-bdb9-66d6a9bcfbd7 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:50.210568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:50.210568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:50.210568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:50.210568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] PATH_INFO: `/v3/users/8f66e068b300440eb108c4c151941279` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:50.212513 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f66e068b300440eb108c4c151941279)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:50.214611 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:50.232351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:50.404837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f66e068b300440eb108c4c151941279'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.405499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f66e068b300440eb108c4c151941279'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.413949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f66e068b300440eb108c4c151941279'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.503695 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f66e068b300440eb108c4c151941279'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.504248 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f66e068b300440eb108c4c151941279'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.522644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1091/2177] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:39:50 2022] DELETE /identity/v3/users/8f66e068b300440eb108c4c151941279 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:39:50.544073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:50.545610 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:50.560368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea5fb0-ffa9-42e0-bb15-8de81e1666af admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:50.621033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:50.621762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:50.622180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:50.622637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] PATH_INFO: `/v3/users/75e5de4fccd243e580a12e2475162451` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:50.623890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e5de4fccd243e580a12e2475162451)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:50.626227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:50.846533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e5de4fccd243e580a12e2475162451'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.847202 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e5de4fccd243e580a12e2475162451'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.915315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e5de4fccd243e580a12e2475162451'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.915872 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e5de4fccd243e580a12e2475162451'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.923699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e5de4fccd243e580a12e2475162451'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:50.941810 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1087/2178] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:39:50 2022] DELETE /identity/v3/users/75e5de4fccd243e580a12e2475162451 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:50.986425 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:50.990826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:51.022645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a73b266-55d1-4c0e-b362-d5ae98c3d884 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.084350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.084989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.085285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.085580 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] PATH_INFO: `/v3/projects/3d87ccde5fe2430b8dc92bfeccdabbcf` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.086719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d87ccde5fe2430b8dc92bfeccdabbcf)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.088642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:51.207939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:51.212337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:51.224602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5289b7a-9bf9-47bb-9b7b-053518586516 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.230581 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:51.235211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d87ccde5fe2430b8dc92bfeccdabbcf', 'request_id': 'req-6d8b9928-bf50-4c78-a4d9-9c25813b9044'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:51.242740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d87ccde5fe2430b8dc92bfeccdabbcf', 'request_id': 'req-6d8b9928-bf50-4c78-a4d9-9c25813b9044'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:51.283741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.284395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.284692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.284953 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] PATH_INFO: `/v3/projects/765785515cdb43c49525c3d1cfa8be6a` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.286144 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=765785515cdb43c49525c3d1cfa8be6a)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.288081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:51.309976 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d87ccde5fe2430b8dc92bfeccdabbcf', 'request_id': 'req-6d8b9928-bf50-4c78-a4d9-9c25813b9044'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:51.322198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] The token cache is being invalidate because project 3d87ccde5fe2430b8dc92bfeccdabbcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:39:51.322489 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:51.323136 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-6d8b9928-bf50-4c78-a4d9-9c25813b9044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:51.324619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1092/2179] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:39:50 2022] DELETE /identity/v3/projects/3d87ccde5fe2430b8dc92bfeccdabbcf => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:51.329156 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c68d61-af74-45fc-95d7-5684184147d6 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.329511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c68d61-af74-45fc-95d7-5684184147d6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.329862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c68d61-af74-45fc-95d7-5684184147d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.350279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-14c68d61-af74-45fc-95d7-5684184147d6 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:51.352107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-14c68d61-af74-45fc-95d7-5684184147d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:51.360310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1093/2180] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:39:51.367857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2cb10-01e6-44ad-80c0-8997aea184de None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.368135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2cb10-01e6-44ad-80c0-8997aea184de None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.368406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2cb10-01e6-44ad-80c0-8997aea184de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.387852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-d8b2cb10-01e6-44ad-80c0-8997aea184de None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:51.389201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d8b2cb10-01e6-44ad-80c0-8997aea184de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:51.394828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '765785515cdb43c49525c3d1cfa8be6a', 'request_id': 'req-4ddc18e2-1cb9-472c-a968-04140ec36b82'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:51.401525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '765785515cdb43c49525c3d1cfa8be6a', 'request_id': 'req-4ddc18e2-1cb9-472c-a968-04140ec36b82'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:51.402860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:51.453885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1094/2181] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:39:51.457837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2cb10-01e6-44ad-80c0-8997aea184de None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:51.467724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2cb10-01e6-44ad-80c0-8997aea184de None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.472450 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d56d61-46b8-4bd0-8eb9-1f99eaf5230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.473051 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d56d61-46b8-4bd0-8eb9-1f99eaf5230a admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.473326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d56d61-46b8-4bd0-8eb9-1f99eaf5230a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.473667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d56d61-46b8-4bd0-8eb9-1f99eaf5230a admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.474270 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d56d61-46b8-4bd0-8eb9-1f99eaf5230a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:39:51.474570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d56d61-46b8-4bd0-8eb9-1f99eaf5230a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.475605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '765785515cdb43c49525c3d1cfa8be6a', 'request_id': 'req-4ddc18e2-1cb9-472c-a968-04140ec36b82'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:51.476791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d56d61-46b8-4bd0-8eb9-1f99eaf5230a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:51.488623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1095/2182] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:39:51.493046 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d56d61-46b8-4bd0-8eb9-1f99eaf5230a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:51.496120 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] The token cache is being invalidate because project 765785515cdb43c49525c3d1cfa8be6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:39:51.496559 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:51.497348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4ddc18e2-1cb9-472c-a968-04140ec36b82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:51.499081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1088/2183] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:39:51 2022] DELETE /identity/v3/projects/765785515cdb43c49525c3d1cfa8be6a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:39:51.504784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d56d61-46b8-4bd0-8eb9-1f99eaf5230a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.508648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803208-9ad3-4e60-abdd-c2cff036a887 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.508929 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803208-9ad3-4e60-abdd-c2cff036a887 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.509202 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803208-9ad3-4e60-abdd-c2cff036a887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.510190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5bfa1-a142-4843-813d-d139ab508c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.510793 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5bfa1-a142-4843-813d-d139ab508c28 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.511069 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5bfa1-a142-4843-813d-d139ab508c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.511337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5bfa1-a142-4843-813d-d139ab508c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.511962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5bfa1-a142-4843-813d-d139ab508c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.513963 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5bfa1-a142-4843-813d-d139ab508c28 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:51.531487 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-cf803208-9ad3-4e60-abdd-c2cff036a887 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:51.532698 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cf803208-9ad3-4e60-abdd-c2cff036a887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:51.540748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1089/2184] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:51.550838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac063814-f41b-44c4-a81e-111d8bfea806 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.551153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac063814-f41b-44c4-a81e-111d8bfea806 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.551427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac063814-f41b-44c4-a81e-111d8bfea806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.571285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-ac063814-f41b-44c4-a81e-111d8bfea806 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:51.572386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ac063814-f41b-44c4-a81e-111d8bfea806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:51.580124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1090/2185] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:39:51.584003 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac063814-f41b-44c4-a81e-111d8bfea806 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:51.594240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac063814-f41b-44c4-a81e-111d8bfea806 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.596926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1096/2186] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:39:51 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:39:51.598839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa4a15-eb65-4f57-9284-ac659d3a5126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.599427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4a15-eb65-4f57-9284-ac659d3a5126 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.599709 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4a15-eb65-4f57-9284-ac659d3a5126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.600039 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa4a15-eb65-4f57-9284-ac659d3a5126 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.600699 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa4a15-eb65-4f57-9284-ac659d3a5126 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:39:51.600961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5bfa1-a142-4843-813d-d139ab508c28 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:51.601079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa4a15-eb65-4f57-9284-ac659d3a5126 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.602668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa4a15-eb65-4f57-9284-ac659d3a5126 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:51.602796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-75f5bfa1-a142-4843-813d-d139ab508c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:51.615176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1091/2187] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:39:51.619325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa4a15-eb65-4f57-9284-ac659d3a5126 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:51.629127 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa4a15-eb65-4f57-9284-ac659d3a5126 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.670732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5bfa1-a142-4843-813d-d139ab508c28 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.675784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a7b7b-fa43-4604-a342-f9c95f447885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.676406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a7b7b-fa43-4604-a342-f9c95f447885 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.676682 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a7b7b-fa43-4604-a342-f9c95f447885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.676950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a7b7b-fa43-4604-a342-f9c95f447885 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.677712 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a7b7b-fa43-4604-a342-f9c95f447885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.679759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a7b7b-fa43-4604-a342-f9c95f447885 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:51.686490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995e5a2-3ba7-4297-b71c-065ebc40969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.687087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995e5a2-3ba7-4297-b71c-065ebc40969c admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.687363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995e5a2-3ba7-4297-b71c-065ebc40969c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.687635 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995e5a2-3ba7-4297-b71c-065ebc40969c admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.688255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995e5a2-3ba7-4297-b71c-065ebc40969c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.690155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0995e5a2-3ba7-4297-b71c-065ebc40969c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:51.795404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1097/2188] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:39:51 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:39:51.799665 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a7b7b-fa43-4604-a342-f9c95f447885 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:51.809187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a7b7b-fa43-4604-a342-f9c95f447885 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.812256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:51.814241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647559c-7389-4654-8573-f8c73ce1628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.814827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647559c-7389-4654-8573-f8c73ce1628d admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.815105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647559c-7389-4654-8573-f8c73ce1628d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.815383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647559c-7389-4654-8573-f8c73ce1628d admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.816014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647559c-7389-4654-8573-f8c73ce1628d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.817602 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647559c-7389-4654-8573-f8c73ce1628d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:51.821005 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1092/2189] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:39:51 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:39:51.826484 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995e5a2-3ba7-4297-b71c-065ebc40969c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:51.831416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1098/2190] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:39:51.832361 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647559c-7389-4654-8573-f8c73ce1628d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:51.842657 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647559c-7389-4654-8573-f8c73ce1628d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.893922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995e5a2-3ba7-4297-b71c-065ebc40969c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.900398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc89f92-f26e-4d94-b3d6-f2f0a90f74b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.901009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc89f92-f26e-4d94-b3d6-f2f0a90f74b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.901289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc89f92-f26e-4d94-b3d6-f2f0a90f74b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.901898 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc89f92-f26e-4d94-b3d6-f2f0a90f74b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.903073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd791d-9c4a-4abd-9434-5c4b8245d88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.903756 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc89f92-f26e-4d94-b3d6-f2f0a90f74b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.903874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd791d-9c4a-4abd-9434-5c4b8245d88d admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.903874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd791d-9c4a-4abd-9434-5c4b8245d88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.903874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd791d-9c4a-4abd-9434-5c4b8245d88d admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.904191 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd791d-9c4a-4abd-9434-5c4b8245d88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.906061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd791d-9c4a-4abd-9434-5c4b8245d88d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:51.906205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc89f92-f26e-4d94-b3d6-f2f0a90f74b3 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:51.908974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:51.917852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1099/2191] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:39:51.921998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd791d-9c4a-4abd-9434-5c4b8245d88d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:51.932443 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd791d-9c4a-4abd-9434-5c4b8245d88d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.937612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602a09f-149c-4445-b535-cd4fcfa93e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.938278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602a09f-149c-4445-b535-cd4fcfa93e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.938633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602a09f-149c-4445-b535-cd4fcfa93e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.938937 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602a09f-149c-4445-b535-cd4fcfa93e72 admin admin] PATH_INFO: `/v3/projects/da34150a958243babf613f18d7784c57/users/16d3c67ef90b41e99788cfde3bc9ccce/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.954018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602a09f-149c-4445-b535-cd4fcfa93e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da34150a958243babf613f18d7784c57, user_id=16d3c67ef90b41e99788cfde3bc9ccce, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.956694 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602a09f-149c-4445-b535-cd4fcfa93e72 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:51.977591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1093/2192] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:39:51 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:39:51.981458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc89f92-f26e-4d94-b3d6-f2f0a90f74b3 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:51.990807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc89f92-f26e-4d94-b3d6-f2f0a90f74b3 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:51.996186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f482e-8a01-4254-84fc-565bdca6340f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:51.996757 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f482e-8a01-4254-84fc-565bdca6340f admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:51.997014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f482e-8a01-4254-84fc-565bdca6340f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:51.997291 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f482e-8a01-4254-84fc-565bdca6340f admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:51.997921 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f482e-8a01-4254-84fc-565bdca6340f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:51.999566 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f482e-8a01-4254-84fc-565bdca6340f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:52.011130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1094/2193] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:39:52.014747 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f482e-8a01-4254-84fc-565bdca6340f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:52.024864 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f482e-8a01-4254-84fc-565bdca6340f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:52.029679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e003f2-52a2-4134-ac1f-f1eccd82551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:52.030491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e003f2-52a2-4134-ac1f-f1eccd82551a admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.030773 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e003f2-52a2-4134-ac1f-f1eccd82551a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.031184 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e003f2-52a2-4134-ac1f-f1eccd82551a admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.031834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e003f2-52a2-4134-ac1f-f1eccd82551a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:52.033454 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e003f2-52a2-4134-ac1f-f1eccd82551a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:52.040731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1100/2194] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:39:51 2022] PUT /identity/v3/projects/da34150a958243babf613f18d7784c57/users/16d3c67ef90b41e99788cfde3bc9ccce/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:39:52.043668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1095/2195] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:39:52.052807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c3cc-dc2f-4219-8541-78173636d89b None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.053108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c3cc-dc2f-4219-8541-78173636d89b None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.053461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4c3cc-dc2f-4219-8541-78173636d89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.057683 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e003f2-52a2-4134-ac1f-f1eccd82551a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:52.067968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e003f2-52a2-4134-ac1f-f1eccd82551a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:52.085054 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-3fd4c3cc-dc2f-4219-8541-78173636d89b None None] MFA Rules not processed for user `16d3c67ef90b41e99788cfde3bc9ccce`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:52.086382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3fd4c3cc-dc2f-4219-8541-78173636d89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:52.131539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea830eb0-4929-41b0-ac29-c6ea665ea50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:52.132210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea830eb0-4929-41b0-ac29-c6ea665ea50c admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.133049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea830eb0-4929-41b0-ac29-c6ea665ea50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.133491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea830eb0-4929-41b0-ac29-c6ea665ea50c admin admin] PATH_INFO: `/v3/projects/b0182ee3a9d14f89a8da298333dbecf7/users/e93e532e6bc44e07888a676219758564/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.150492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea830eb0-4929-41b0-ac29-c6ea665ea50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0182ee3a9d14f89a8da298333dbecf7, user_id=e93e532e6bc44e07888a676219758564, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:52.153147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea830eb0-4929-41b0-ac29-c6ea665ea50c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:52.166296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1101/2196] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:39:52.170973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4c3cc-dc2f-4219-8541-78173636d89b None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:52.181671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4c3cc-dc2f-4219-8541-78173636d89b None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:52.186180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71208a-00a1-4186-88b9-665b081f5df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:52.186774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71208a-00a1-4186-88b9-665b081f5df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.187053 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71208a-00a1-4186-88b9-665b081f5df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.187318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71208a-00a1-4186-88b9-665b081f5df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.187945 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71208a-00a1-4186-88b9-665b081f5df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:52.189810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71208a-00a1-4186-88b9-665b081f5df5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:52.223757 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1096/2197] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:39:52 2022] PUT /identity/v3/projects/b0182ee3a9d14f89a8da298333dbecf7/users/e93e532e6bc44e07888a676219758564/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:52.240741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbe2d8-6eaa-4283-b5d7-68f85cf77345 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.241024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbe2d8-6eaa-4283-b5d7-68f85cf77345 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.241297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbe2d8-6eaa-4283-b5d7-68f85cf77345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.269023 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b6cbe2d8-6eaa-4283-b5d7-68f85cf77345 None None] MFA Rules not processed for user `e93e532e6bc44e07888a676219758564`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:52.270242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b6cbe2d8-6eaa-4283-b5d7-68f85cf77345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:52.287820 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1102/2198] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:39:52 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:39:52.292606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71208a-00a1-4186-88b9-665b081f5df5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:52.308918 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:52.333604 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1097/2199] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:39:52.344643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbe2d8-6eaa-4283-b5d7-68f85cf77345 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:52.345973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b6cbe2d8-6eaa-4283-b5d7-68f85cf77345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:52.355628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71208a-00a1-4186-88b9-665b081f5df5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:52.360289 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f314293-932d-430f-a760-ecd41e0cfb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:52.360887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f314293-932d-430f-a760-ecd41e0cfb3a admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.361159 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f314293-932d-430f-a760-ecd41e0cfb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.361425 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f314293-932d-430f-a760-ecd41e0cfb3a admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.362096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f314293-932d-430f-a760-ecd41e0cfb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:52.363990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f314293-932d-430f-a760-ecd41e0cfb3a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:52.419556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbe2d8-6eaa-4283-b5d7-68f85cf77345 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:52.455834 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1103/2200] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:39:52 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:39:52.456915 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f314293-932d-430f-a760-ecd41e0cfb3a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:52.466479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f314293-932d-430f-a760-ecd41e0cfb3a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:52.471300 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3f554-0aa4-4200-9fe8-abdcb87561d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:52.471887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3f554-0aa4-4200-9fe8-abdcb87561d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.472165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3f554-0aa4-4200-9fe8-abdcb87561d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.472440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3f554-0aa4-4200-9fe8-abdcb87561d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.473163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b3f554-0aa4-4200-9fe8-abdcb87561d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:52.474796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b3f554-0aa4-4200-9fe8-abdcb87561d7 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:52.481135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c5596-d09a-4ec8-9d5b-f01541339714 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:52.481740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c5596-d09a-4ec8-9d5b-f01541339714 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.482165 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c5596-d09a-4ec8-9d5b-f01541339714 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.482443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c5596-d09a-4ec8-9d5b-f01541339714 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.485808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1104/2201] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:39:52.489501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3f554-0aa4-4200-9fe8-abdcb87561d7 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:52.499753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3f554-0aa4-4200-9fe8-abdcb87561d7 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:52.504637 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebf048-e80b-4126-9502-555f3c6fbe65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:52.505263 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebf048-e80b-4126-9502-555f3c6fbe65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.505537 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebf048-e80b-4126-9502-555f3c6fbe65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.505803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebf048-e80b-4126-9502-555f3c6fbe65 admin admin] PATH_INFO: `/v3/projects/c7cc52515e5a4efdb1e4d17fb1f1fa66/users/5aa19e33b7934a468caed42f86f07a16/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.520601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ebf048-e80b-4126-9502-555f3c6fbe65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7cc52515e5a4efdb1e4d17fb1f1fa66, user_id=5aa19e33b7934a468caed42f86f07a16, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:52.523704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ebf048-e80b-4126-9502-555f3c6fbe65 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:52.526834 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:52.540313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c5596-d09a-4ec8-9d5b-f01541339714 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:52.541906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c5596-d09a-4ec8-9d5b-f01541339714 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:52.550684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1098/2202] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:39:52.635626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1105/2203] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:39:52 2022] PUT /identity/v3/projects/c7cc52515e5a4efdb1e4d17fb1f1fa66/users/5aa19e33b7934a468caed42f86f07a16/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:39:52.647891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c5596-d09a-4ec8-9d5b-f01541339714 service nova] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:52.659110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c5596-d09a-4ec8-9d5b-f01541339714 service nova] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:52.669246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:52.717697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2030b6a-aab8-4a1a-af9f-ddaa3a81621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:52.718414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2030b6a-aab8-4a1a-af9f-ddaa3a81621d admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.718715 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2030b6a-aab8-4a1a-af9f-ddaa3a81621d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.719056 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2030b6a-aab8-4a1a-af9f-ddaa3a81621d admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.719727 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2030b6a-aab8-4a1a-af9f-ddaa3a81621d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:52.721333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2030b6a-aab8-4a1a-af9f-ddaa3a81621d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:52.732188 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1099/2204] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:39:52.736410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebf048-e80b-4126-9502-555f3c6fbe65 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:52.746629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebf048-e80b-4126-9502-555f3c6fbe65 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:52.751427 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73418d71-78c7-4d7f-819b-5ae66d483807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:52.752067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73418d71-78c7-4d7f-819b-5ae66d483807 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.752341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73418d71-78c7-4d7f-819b-5ae66d483807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.752609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73418d71-78c7-4d7f-819b-5ae66d483807 admin admin] PATH_INFO: `/v3/projects/c7cc52515e5a4efdb1e4d17fb1f1fa66/users/5aa19e33b7934a468caed42f86f07a16/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.754524 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73418d71-78c7-4d7f-819b-5ae66d483807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7cc52515e5a4efdb1e4d17fb1f1fa66, user_id=5aa19e33b7934a468caed42f86f07a16, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:52.757105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73418d71-78c7-4d7f-819b-5ae66d483807 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:52.767638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1106/2205] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:39:52 2022] PUT /identity/v3/projects/c7cc52515e5a4efdb1e4d17fb1f1fa66/users/5aa19e33b7934a468caed42f86f07a16/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:52.780778 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bd736-d02d-42b7-b1c6-100aa0ab714e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.781064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bd736-d02d-42b7-b1c6-100aa0ab714e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.781336 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bd736-d02d-42b7-b1c6-100aa0ab714e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.809001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b10bd736-d02d-42b7-b1c6-100aa0ab714e None None] MFA Rules not processed for user `5aa19e33b7934a468caed42f86f07a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:52.817229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b10bd736-d02d-42b7-b1c6-100aa0ab714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:52.878003 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1100/2206] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:52.889425 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73418d71-78c7-4d7f-819b-5ae66d483807 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:52.895973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-73418d71-78c7-4d7f-819b-5ae66d483807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:52.947888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73418d71-78c7-4d7f-819b-5ae66d483807 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:52.951495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ddb9b-552a-493a-8bc1-8339f3a04f84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:52.952087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ddb9b-552a-493a-8bc1-8339f3a04f84 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:52.952361 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ddb9b-552a-493a-8bc1-8339f3a04f84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:52.952625 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ddb9b-552a-493a-8bc1-8339f3a04f84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:52.961979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ddb9b-552a-493a-8bc1-8339f3a04f84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:52.963408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ddb9b-552a-493a-8bc1-8339f3a04f84 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:52.972667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1107/2207] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:39:53.998312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353e265-74f9-4742-a3c4-b35eb77b1620 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:54.002366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353e265-74f9-4742-a3c4-b35eb77b1620 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:54.002643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353e265-74f9-4742-a3c4-b35eb77b1620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:54.028087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-0353e265-74f9-4742-a3c4-b35eb77b1620 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:54.029415 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0353e265-74f9-4742-a3c4-b35eb77b1620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:54.069984 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ddb9b-552a-493a-8bc1-8339f3a04f84 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:54.092818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1101/2208] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:39:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:54.108500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353e265-74f9-4742-a3c4-b35eb77b1620 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:54.110169 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0353e265-74f9-4742-a3c4-b35eb77b1620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:54.151898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ddb9b-552a-493a-8bc1-8339f3a04f84 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:54.157063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:54.157933 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:54.158408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:54.158784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] PATH_INFO: `/v3/users/16d3c67ef90b41e99788cfde3bc9ccce` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:54.160044 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d3c67ef90b41e99788cfde3bc9ccce)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:54.162151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:54.165853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:54.198573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353e265-74f9-4742-a3c4-b35eb77b1620 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:54.203526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73c3a1-5933-4a82-89d5-89d7ec98c005 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:54.204177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73c3a1-5933-4a82-89d5-89d7ec98c005 service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:54.204461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73c3a1-5933-4a82-89d5-89d7ec98c005 service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:54.204739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73c3a1-5933-4a82-89d5-89d7ec98c005 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:54.213363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73c3a1-5933-4a82-89d5-89d7ec98c005 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:54.215092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73c3a1-5933-4a82-89d5-89d7ec98c005 service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:54.223671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1102/2209] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:39:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:39:54.341596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d3c67ef90b41e99788cfde3bc9ccce'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:54.342439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d3c67ef90b41e99788cfde3bc9ccce'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:54.352281 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d3c67ef90b41e99788cfde3bc9ccce'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:54.436589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d3c67ef90b41e99788cfde3bc9ccce'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:54.436589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d3c67ef90b41e99788cfde3bc9ccce'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:54.464208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1108/2210] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:39:54 2022] DELETE /identity/v3/users/16d3c67ef90b41e99788cfde3bc9ccce => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:39:54.468578 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb70b9-2d26-42fa-b0b1-1ff9ae6810f5 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:54.468782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb70b9-2d26-42fa-b0b1-1ff9ae6810f5 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:54.469057 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb70b9-2d26-42fa-b0b1-1ff9ae6810f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:54.500413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8ddb70b9-2d26-42fa-b0b1-1ff9ae6810f5 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:54.501892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8ddb70b9-2d26-42fa-b0b1-1ff9ae6810f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:54.623697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1103/2211] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:39:54.635790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:54.637833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:54.647105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:54.729863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72899354-eb53-4930-9db7-c9adf461d7d1 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:54.729863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017d7a2-564f-4c75-9d10-8b1bbf0b4fd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:54.729863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017d7a2-564f-4c75-9d10-8b1bbf0b4fd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:54.729863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017d7a2-564f-4c75-9d10-8b1bbf0b4fd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:54.730382 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017d7a2-564f-4c75-9d10-8b1bbf0b4fd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:54.735889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017d7a2-564f-4c75-9d10-8b1bbf0b4fd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:54.737466 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017d7a2-564f-4c75-9d10-8b1bbf0b4fd3 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:54.748297 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1109/2212] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:39:55.236960 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb70b9-2d26-42fa-b0b1-1ff9ae6810f5 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:55.239071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8ddb70b9-2d26-42fa-b0b1-1ff9ae6810f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:55.254453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb70b9-2d26-42fa-b0b1-1ff9ae6810f5 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:55.265021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:55.265021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:55.265021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:55.265021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] PATH_INFO: `/v3/projects/da34150a958243babf613f18d7784c57` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:55.265386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da34150a958243babf613f18d7784c57)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:55.267716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:55.382758 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da34150a958243babf613f18d7784c57', 'request_id': 'req-b63e62f9-a46a-4f5d-b915-4b434da0e713'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:55.389711 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da34150a958243babf613f18d7784c57', 'request_id': 'req-b63e62f9-a46a-4f5d-b915-4b434da0e713'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:55.451620 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da34150a958243babf613f18d7784c57', 'request_id': 'req-b63e62f9-a46a-4f5d-b915-4b434da0e713'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:55.465541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] The token cache is being invalidate because project da34150a958243babf613f18d7784c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:39:55.466266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:55.467006 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:55.468841 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1104/2213] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:39:55 2022] DELETE /identity/v3/projects/da34150a958243babf613f18d7784c57 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:55.472651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017d7a2-564f-4c75-9d10-8b1bbf0b4fd3 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:55.483557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017d7a2-564f-4c75-9d10-8b1bbf0b4fd3 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:55.551741 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:55.552545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:55.552824 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:55.553096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] PATH_INFO: `/v3/users/5aa19e33b7934a468caed42f86f07a16` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:55.554917 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aa19e33b7934a468caed42f86f07a16)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:55.556805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:55.774866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aa19e33b7934a468caed42f86f07a16'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:55.775784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aa19e33b7934a468caed42f86f07a16'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:55.790761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aa19e33b7934a468caed42f86f07a16'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:55.852474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aa19e33b7934a468caed42f86f07a16'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:55.853305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aa19e33b7934a468caed42f86f07a16'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:55.881108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1110/2214] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:39:55 2022] DELETE /identity/v3/users/5aa19e33b7934a468caed42f86f07a16 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:56.094937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:56.101500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:56.135081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e62f9-a46a-4f5d-b915-4b434da0e713 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:56.229167 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:56.241064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d29b2-cf5c-41c2-8c20-fd3377c2e26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:56.241759 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d29b2-cf5c-41c2-8c20-fd3377c2e26e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:56.242002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d29b2-cf5c-41c2-8c20-fd3377c2e26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:56.242371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d29b2-cf5c-41c2-8c20-fd3377c2e26e admin admin] PATH_INFO: `/v3/users/e93e532e6bc44e07888a676219758564` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:56.243653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d29b2-cf5c-41c2-8c20-fd3377c2e26e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93e532e6bc44e07888a676219758564)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:56.247832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d29b2-cf5c-41c2-8c20-fd3377c2e26e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:56.432747 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:56.443771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:56.465623 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d962322-453e-4846-9921-1f50a9c2b0db admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:56.466826 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-798d29b2-cf5c-41c2-8c20-fd3377c2e26e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93e532e6bc44e07888a676219758564'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:56.467793 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-798d29b2-cf5c-41c2-8c20-fd3377c2e26e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93e532e6bc44e07888a676219758564'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:56.485699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:56.486380 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:56.486654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:56.486922 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] PATH_INFO: `/v3/projects/c7cc52515e5a4efdb1e4d17fb1f1fa66` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:56.489853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7cc52515e5a4efdb1e4d17fb1f1fa66)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:56.489988 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:56.518093 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:56.549838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-798d29b2-cf5c-41c2-8c20-fd3377c2e26e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93e532e6bc44e07888a676219758564'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:56.550551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-798d29b2-cf5c-41c2-8c20-fd3377c2e26e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93e532e6bc44e07888a676219758564'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:56.559458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-798d29b2-cf5c-41c2-8c20-fd3377c2e26e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93e532e6bc44e07888a676219758564'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:56.588438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1105/2215] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:39:56 2022] DELETE /identity/v3/users/e93e532e6bc44e07888a676219758564 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:56.589924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957f1d4-027f-4218-bb05-6089aac4c51e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:56.590236 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957f1d4-027f-4218-bb05-6089aac4c51e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:56.590509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957f1d4-027f-4218-bb05-6089aac4c51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:56.600859 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7cc52515e5a4efdb1e4d17fb1f1fa66', 'request_id': 'req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:56.608539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7cc52515e5a4efdb1e4d17fb1f1fa66', 'request_id': 'req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:56.617389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-5957f1d4-027f-4218-bb05-6089aac4c51e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:56.618643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5957f1d4-027f-4218-bb05-6089aac4c51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:56.695990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1106/2216] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:56.703358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7cc52515e5a4efdb1e4d17fb1f1fa66', 'request_id': 'req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:56.706190 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957f1d4-027f-4218-bb05-6089aac4c51e None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:56.709618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5957f1d4-027f-4218-bb05-6089aac4c51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:56.730431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] The token cache is being invalidate because project c7cc52515e5a4efdb1e4d17fb1f1fa66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:39:56.730942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:56.736596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-620e34c4-b509-4f1a-b4cd-2aaabcd7778d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:56.738572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1111/2217] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:39:56 2022] DELETE /identity/v3/projects/c7cc52515e5a4efdb1e4d17fb1f1fa66 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:39:56.744250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6174b-fed6-4723-8b50-fc6f02efccaa None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:56.750508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6174b-fed6-4723-8b50-fc6f02efccaa None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:56.750508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6174b-fed6-4723-8b50-fc6f02efccaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:56.797698 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-b0a6174b-fed6-4723-8b50-fc6f02efccaa None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:56.798979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b0a6174b-fed6-4723-8b50-fc6f02efccaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:56.809056 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1112/2218] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:56.816867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698d957-44af-430d-bc5b-161a6b78c422 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:56.817220 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698d957-44af-430d-bc5b-161a6b78c422 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:56.817687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698d957-44af-430d-bc5b-161a6b78c422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:56.835232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957f1d4-027f-4218-bb05-6089aac4c51e None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:56.843123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c423151-9da7-4017-8eff-195dd373fb35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:56.843840 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c423151-9da7-4017-8eff-195dd373fb35 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:56.864684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c423151-9da7-4017-8eff-195dd373fb35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:56.864684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c423151-9da7-4017-8eff-195dd373fb35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:56.864684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1c423151-9da7-4017-8eff-195dd373fb35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:56.865170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-1698d957-44af-430d-bc5b-161a6b78c422 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:56.865170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1698d957-44af-430d-bc5b-161a6b78c422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:56.868050 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1113/2219] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:56.870207 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c423151-9da7-4017-8eff-195dd373fb35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:56.872804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698d957-44af-430d-bc5b-161a6b78c422 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:56.874304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c423151-9da7-4017-8eff-195dd373fb35 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:56.885408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698d957-44af-430d-bc5b-161a6b78c422 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:56.888928 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1107/2220] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:39:56.891526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d68b5c-1e7d-41d8-a98b-6b4f616a012c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:56.898732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d68b5c-1e7d-41d8-a98b-6b4f616a012c admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:56.898732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d68b5c-1e7d-41d8-a98b-6b4f616a012c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:56.898732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d68b5c-1e7d-41d8-a98b-6b4f616a012c admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:56.898732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d68b5c-1e7d-41d8-a98b-6b4f616a012c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:39:56.898732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d68b5c-1e7d-41d8-a98b-6b4f616a012c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:56.898732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d68b5c-1e7d-41d8-a98b-6b4f616a012c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:56.914057 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1114/2221] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:39:56.919629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c423151-9da7-4017-8eff-195dd373fb35 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:56.932740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c423151-9da7-4017-8eff-195dd373fb35 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:56.945563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b15408-f0a9-4d59-a694-4bfe6d96237b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:56.950668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b15408-f0a9-4d59-a694-4bfe6d96237b admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:56.950668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b15408-f0a9-4d59-a694-4bfe6d96237b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:56.950668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b15408-f0a9-4d59-a694-4bfe6d96237b admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:56.950668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b15408-f0a9-4d59-a694-4bfe6d96237b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:56.953060 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b15408-f0a9-4d59-a694-4bfe6d96237b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:56.992446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:57.094087 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1108/2222] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:39:56 2022] POST /identity/v3/projects => generated 407 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:39:57.109460 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d68b5c-1e7d-41d8-a98b-6b4f616a012c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:57.122115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d68b5c-1e7d-41d8-a98b-6b4f616a012c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:57.206116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e52b39-f43d-4d7d-a3b5-e067d0dcb45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:57.206816 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e52b39-f43d-4d7d-a3b5-e067d0dcb45d admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:57.207035 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e52b39-f43d-4d7d-a3b5-e067d0dcb45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:57.207287 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e52b39-f43d-4d7d-a3b5-e067d0dcb45d admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:57.208294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e52b39-f43d-4d7d-a3b5-e067d0dcb45d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:57.210004 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e52b39-f43d-4d7d-a3b5-e067d0dcb45d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:57.325400 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1115/2223] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:39:57 2022] POST /identity/v3/users => generated 304 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:39:57.326732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b15408-f0a9-4d59-a694-4bfe6d96237b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:57.339547 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b15408-f0a9-4d59-a694-4bfe6d96237b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:57.355385 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e004f1-235e-4bfa-b568-be2724b91e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:57.361907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e004f1-235e-4bfa-b568-be2724b91e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:57.361907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e004f1-235e-4bfa-b568-be2724b91e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:57.361907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e004f1-235e-4bfa-b568-be2724b91e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:57.361907 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e004f1-235e-4bfa-b568-be2724b91e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:57.371686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e004f1-235e-4bfa-b568-be2724b91e41 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:57.391663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1109/2224] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:39:57.397816 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e52b39-f43d-4d7d-a3b5-e067d0dcb45d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:57.412173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e52b39-f43d-4d7d-a3b5-e067d0dcb45d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:57.419844 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b828052-a309-4ed7-b192-48eaf155c42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:57.420560 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b828052-a309-4ed7-b192-48eaf155c42d admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:57.421016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b828052-a309-4ed7-b192-48eaf155c42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:57.421397 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b828052-a309-4ed7-b192-48eaf155c42d admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:57.422245 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b828052-a309-4ed7-b192-48eaf155c42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:57.424577 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b828052-a309-4ed7-b192-48eaf155c42d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:57.444323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1116/2225] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:39:57.446451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e004f1-235e-4bfa-b568-be2724b91e41 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:57.457607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e004f1-235e-4bfa-b568-be2724b91e41 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:57.462492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1c1fc-4831-4f88-bfd8-b6a10588fc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:57.463340 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1c1fc-4831-4f88-bfd8-b6a10588fc62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:57.463717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1c1fc-4831-4f88-bfd8-b6a10588fc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:57.464081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1c1fc-4831-4f88-bfd8-b6a10588fc62 admin admin] PATH_INFO: `/v3/projects/92d8ab995dac41d09c33a5e2f0c668dd/users/66ba9c37867c4ece8662d34b4a642a41/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:57.482427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1c1fc-4831-4f88-bfd8-b6a10588fc62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d8ab995dac41d09c33a5e2f0c668dd, user_id=66ba9c37867c4ece8662d34b4a642a41, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:57.485367 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1c1fc-4831-4f88-bfd8-b6a10588fc62 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:57.548500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1110/2226] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:39:57 2022] PUT /identity/v3/projects/92d8ab995dac41d09c33a5e2f0c668dd/users/66ba9c37867c4ece8662d34b4a642a41/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:57.563662 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dd4dd-5367-495e-a913-7652fc1520cc None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:57.564175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dd4dd-5367-495e-a913-7652fc1520cc None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:57.564276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dd4dd-5367-495e-a913-7652fc1520cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:57.597908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2f6dd4dd-5367-495e-a913-7652fc1520cc None None] MFA Rules not processed for user `66ba9c37867c4ece8662d34b4a642a41`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:57.599323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2f6dd4dd-5367-495e-a913-7652fc1520cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:57.650942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:57.668931 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1117/2227] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:57 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:57.734560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1c1fc-4831-4f88-bfd8-b6a10588fc62 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:57.778406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1c1fc-4831-4f88-bfd8-b6a10588fc62 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:57.782982 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec0441-8ade-4125-bf8c-caad82d34ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:57.783573 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec0441-8ade-4125-bf8c-caad82d34ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:57.783845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec0441-8ade-4125-bf8c-caad82d34ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:57.784118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec0441-8ade-4125-bf8c-caad82d34ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:57.784736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec0441-8ade-4125-bf8c-caad82d34ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:57.786691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec0441-8ade-4125-bf8c-caad82d34ab3 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:58.064556 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6dd4dd-5367-495e-a913-7652fc1520cc None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:58.066336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2f6dd4dd-5367-495e-a913-7652fc1520cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:58.081339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6dd4dd-5367-495e-a913-7652fc1520cc None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:58.086850 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:58.087529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.087883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.088063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] PATH_INFO: `/v3/projects/b0182ee3a9d14f89a8da298333dbecf7` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.089202 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0182ee3a9d14f89a8da298333dbecf7)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:58.095242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:58.107455 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1111/2228] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:39:57 2022] POST /identity/v3/projects => generated 407 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:39:58.114789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec0441-8ade-4125-bf8c-caad82d34ab3 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:58.175748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec0441-8ade-4125-bf8c-caad82d34ab3 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:58.181062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6be9e4-11a4-406a-a835-3f9360b10f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:58.182018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6be9e4-11a4-406a-a835-3f9360b10f78 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.182018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6be9e4-11a4-406a-a835-3f9360b10f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.182366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6be9e4-11a4-406a-a835-3f9360b10f78 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.183447 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6be9e4-11a4-406a-a835-3f9360b10f78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:58.185474 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6be9e4-11a4-406a-a835-3f9360b10f78 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:58.227338 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0182ee3a9d14f89a8da298333dbecf7', 'request_id': 'req-b8278b27-a141-4b04-83bf-8ed0cd169047'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:58.236176 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0182ee3a9d14f89a8da298333dbecf7', 'request_id': 'req-b8278b27-a141-4b04-83bf-8ed0cd169047'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:58.281426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1112/2229] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:39:58 2022] POST /identity/v3/users => generated 310 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:39:58.285161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6be9e4-11a4-406a-a835-3f9360b10f78 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:58.297185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0182ee3a9d14f89a8da298333dbecf7', 'request_id': 'req-b8278b27-a141-4b04-83bf-8ed0cd169047'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:58.299969 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6be9e4-11a4-406a-a835-3f9360b10f78 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:58.311269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] The token cache is being invalidate because project b0182ee3a9d14f89a8da298333dbecf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:39:58.311531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:58.312459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b8278b27-a141-4b04-83bf-8ed0cd169047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:58.314279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1118/2230] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:39:58 2022] DELETE /identity/v3/projects/b0182ee3a9d14f89a8da298333dbecf7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:58.319481 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d861c-0b11-4816-b820-c819fe9f6f65 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.319903 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d861c-0b11-4816-b820-c819fe9f6f65 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.320295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d861c-0b11-4816-b820-c819fe9f6f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.342406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:58.347882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-987d861c-0b11-4816-b820-c819fe9f6f65 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:58.349256 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-987d861c-0b11-4816-b820-c819fe9f6f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:58.376542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014bb0c6-7490-49d0-8a3a-b1042a4297e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:58.377208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bb0c6-7490-49d0-8a3a-b1042a4297e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.377491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bb0c6-7490-49d0-8a3a-b1042a4297e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.377788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bb0c6-7490-49d0-8a3a-b1042a4297e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.378863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014bb0c6-7490-49d0-8a3a-b1042a4297e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:58.382032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014bb0c6-7490-49d0-8a3a-b1042a4297e5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:58.396468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1113/2231] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:39:58.399706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014bb0c6-7490-49d0-8a3a-b1042a4297e5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:58.401739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-014bb0c6-7490-49d0-8a3a-b1042a4297e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:58.419671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014bb0c6-7490-49d0-8a3a-b1042a4297e5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:58.425517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b8bbc-f0e7-4fa2-a5b3-56e46ecb0bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:58.427244 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b8bbc-f0e7-4fa2-a5b3-56e46ecb0bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.427676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b8bbc-f0e7-4fa2-a5b3-56e46ecb0bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.428173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b8bbc-f0e7-4fa2-a5b3-56e46ecb0bbf admin admin] PATH_INFO: `/v3/projects/18273080be944c70957d8ea9d96cb359/users/bec2f6b29ac6413f9316fa35dc40bba5/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.441528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1119/2232] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:58.449565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5016a2-3aa9-4154-bfdf-c1ad8ab16671 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.449845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5016a2-3aa9-4154-bfdf-c1ad8ab16671 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.450149 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5016a2-3aa9-4154-bfdf-c1ad8ab16671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.454705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b8bbc-f0e7-4fa2-a5b3-56e46ecb0bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=18273080be944c70957d8ea9d96cb359, user_id=bec2f6b29ac6413f9316fa35dc40bba5, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:58.458494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b8bbc-f0e7-4fa2-a5b3-56e46ecb0bbf admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:58.475254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-3c5016a2-3aa9-4154-bfdf-c1ad8ab16671 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:58.477008 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3c5016a2-3aa9-4154-bfdf-c1ad8ab16671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:58.502311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1120/2233] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:58.507502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5016a2-3aa9-4154-bfdf-c1ad8ab16671 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:58.520035 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5016a2-3aa9-4154-bfdf-c1ad8ab16671 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:58.529421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53716eac-c1d2-43c2-9cf1-9d08e5742bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:58.531280 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53716eac-c1d2-43c2-9cf1-9d08e5742bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.535337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53716eac-c1d2-43c2-9cf1-9d08e5742bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.535337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53716eac-c1d2-43c2-9cf1-9d08e5742bf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.535337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53716eac-c1d2-43c2-9cf1-9d08e5742bf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:39:58.535337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53716eac-c1d2-43c2-9cf1-9d08e5742bf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:58.535691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1114/2234] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:39:58 2022] PUT /identity/v3/projects/18273080be944c70957d8ea9d96cb359/users/bec2f6b29ac6413f9316fa35dc40bba5/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:39:58.539754 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53716eac-c1d2-43c2-9cf1-9d08e5742bf7 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:58.545553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b8bbc-f0e7-4fa2-a5b3-56e46ecb0bbf admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:58.568042 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1121/2235] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:39:58.570693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53716eac-c1d2-43c2-9cf1-9d08e5742bf7 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:58.602089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53716eac-c1d2-43c2-9cf1-9d08e5742bf7 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:58.649226 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b8bbc-f0e7-4fa2-a5b3-56e46ecb0bbf admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:58.658648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545501a-1e26-4ac6-8221-8c281151c368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:58.659685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545501a-1e26-4ac6-8221-8c281151c368 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.660060 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545501a-1e26-4ac6-8221-8c281151c368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.660432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545501a-1e26-4ac6-8221-8c281151c368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.661367 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545501a-1e26-4ac6-8221-8c281151c368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:58.664328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545501a-1e26-4ac6-8221-8c281151c368 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:58.675124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:58.678434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1115/2236] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:39:58.683682 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545501a-1e26-4ac6-8221-8c281151c368 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:58.691369 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128267b-1ff6-4aa1-99f9-960809a9ad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:58.692228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128267b-1ff6-4aa1-99f9-960809a9ad23 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.692667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128267b-1ff6-4aa1-99f9-960809a9ad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.695590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128267b-1ff6-4aa1-99f9-960809a9ad23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.695590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128267b-1ff6-4aa1-99f9-960809a9ad23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:58.695816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545501a-1e26-4ac6-8221-8c281151c368 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:58.697136 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a128267b-1ff6-4aa1-99f9-960809a9ad23 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:58.699945 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16013c-1a59-4a3b-bfd3-fcc3852cd1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:58.700776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16013c-1a59-4a3b-bfd3-fcc3852cd1ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.703164 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16013c-1a59-4a3b-bfd3-fcc3852cd1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.703164 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16013c-1a59-4a3b-bfd3-fcc3852cd1ab admin admin] PATH_INFO: `/v3/projects/18273080be944c70957d8ea9d96cb359/users/bec2f6b29ac6413f9316fa35dc40bba5/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.704797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16013c-1a59-4a3b-bfd3-fcc3852cd1ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=18273080be944c70957d8ea9d96cb359, user_id=bec2f6b29ac6413f9316fa35dc40bba5, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:58.708011 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16013c-1a59-4a3b-bfd3-fcc3852cd1ab admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:58.720086 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1116/2237] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:39:58 2022] PUT /identity/v3/projects/18273080be944c70957d8ea9d96cb359/users/bec2f6b29ac6413f9316fa35dc40bba5/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:58.730609 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629785ff-b21d-4cbc-8898-6b3f442a2177 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.730996 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629785ff-b21d-4cbc-8898-6b3f442a2177 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.731370 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629785ff-b21d-4cbc-8898-6b3f442a2177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.765477 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-629785ff-b21d-4cbc-8898-6b3f442a2177 None None] MFA Rules not processed for user `bec2f6b29ac6413f9316fa35dc40bba5`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:58.766964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-629785ff-b21d-4cbc-8898-6b3f442a2177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:58.819346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1122/2238] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:39:58 2022] POST /identity/v3/projects => generated 373 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:39:58.822169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128267b-1ff6-4aa1-99f9-960809a9ad23 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:58.861612 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1117/2239] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:58 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:39:58.864014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629785ff-b21d-4cbc-8898-6b3f442a2177 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:58.865536 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-629785ff-b21d-4cbc-8898-6b3f442a2177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:58.885719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:58.933593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128267b-1ff6-4aa1-99f9-960809a9ad23 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:58.939626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e0f08-a779-4498-8881-1657be9376d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:58.940225 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e0f08-a779-4498-8881-1657be9376d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.940502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e0f08-a779-4498-8881-1657be9376d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.940771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e0f08-a779-4498-8881-1657be9376d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:58.941214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629785ff-b21d-4cbc-8898-6b3f442a2177 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:58.941410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e0f08-a779-4498-8881-1657be9376d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:58.943358 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e0f08-a779-4498-8881-1657be9376d2 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:58.996509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d219a0-9e9f-48d0-a1aa-59585a7e0f14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:58.997118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d219a0-9e9f-48d0-a1aa-59585a7e0f14 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:58.997391 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d219a0-9e9f-48d0-a1aa-59585a7e0f14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:58.997660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d219a0-9e9f-48d0-a1aa-59585a7e0f14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:59.037655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1123/2240] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:39:58 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:39:59.039016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e0f08-a779-4498-8881-1657be9376d2 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:59.055386 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e0f08-a779-4498-8881-1657be9376d2 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:59.061168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5d2c9-1e7d-4a2c-a6c6-3af7eb48a76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:59.061875 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5d2c9-1e7d-4a2c-a6c6-3af7eb48a76d admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:59.062220 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5d2c9-1e7d-4a2c-a6c6-3af7eb48a76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:59.062472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5d2c9-1e7d-4a2c-a6c6-3af7eb48a76d admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:59.063219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5d2c9-1e7d-4a2c-a6c6-3af7eb48a76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:59.064895 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5d2c9-1e7d-4a2c-a6c6-3af7eb48a76d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:59.077274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1124/2241] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:39:59.079519 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d219a0-9e9f-48d0-a1aa-59585a7e0f14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:59.081231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d219a0-9e9f-48d0-a1aa-59585a7e0f14 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:59.081454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5d2c9-1e7d-4a2c-a6c6-3af7eb48a76d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:59.096174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5d2c9-1e7d-4a2c-a6c6-3af7eb48a76d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:59.096665 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1118/2242] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:58 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:39:59.111156 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409850d8-3ca8-4f69-9f3f-2b70003dee51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:59.112591 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409850d8-3ca8-4f69-9f3f-2b70003dee51 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:59.113053 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409850d8-3ca8-4f69-9f3f-2b70003dee51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:59.113674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409850d8-3ca8-4f69-9f3f-2b70003dee51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:59.114560 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409850d8-3ca8-4f69-9f3f-2b70003dee51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:59.116429 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409850d8-3ca8-4f69-9f3f-2b70003dee51 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:59.133552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1125/2243] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:39:59.135772 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d219a0-9e9f-48d0-a1aa-59585a7e0f14 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:59.150138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d219a0-9e9f-48d0-a1aa-59585a7e0f14 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:59.156026 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e82028-59d9-4d0f-9446-0f095a3474a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:59.156945 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e82028-59d9-4d0f-9446-0f095a3474a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:59.157387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e82028-59d9-4d0f-9446-0f095a3474a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:59.158009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e82028-59d9-4d0f-9446-0f095a3474a3 admin admin] PATH_INFO: `/v3/projects/32aa73aeb0724ddbbe498f5ca95ff88e/users/2027b8eab6e140b796a4d20a015704d7/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:59.179679 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e82028-59d9-4d0f-9446-0f095a3474a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32aa73aeb0724ddbbe498f5ca95ff88e, user_id=2027b8eab6e140b796a4d20a015704d7, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:59.182486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e82028-59d9-4d0f-9446-0f095a3474a3 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:59.260292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1119/2244] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:39:59 2022] PUT /identity/v3/projects/32aa73aeb0724ddbbe498f5ca95ff88e/users/2027b8eab6e140b796a4d20a015704d7/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:39:59.270808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc74415-9a02-4a3d-a179-f019c265ac28 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:59.271333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc74415-9a02-4a3d-a179-f019c265ac28 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:59.271691 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc74415-9a02-4a3d-a179-f019c265ac28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:59.304636 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-4fc74415-9a02-4a3d-a179-f019c265ac28 None None] MFA Rules not processed for user `2027b8eab6e140b796a4d20a015704d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:39:59.305872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4fc74415-9a02-4a3d-a179-f019c265ac28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:59.393564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1126/2245] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:39:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:39:59.409409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e82028-59d9-4d0f-9446-0f095a3474a3 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:59.487555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e82028-59d9-4d0f-9446-0f095a3474a3 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:59.494644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c6ec4-7057-4221-b6ef-2422af1ade07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:59.495591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c6ec4-7057-4221-b6ef-2422af1ade07 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:59.496272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c6ec4-7057-4221-b6ef-2422af1ade07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:59.496712 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c6ec4-7057-4221-b6ef-2422af1ade07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:59.512280 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c6ec4-7057-4221-b6ef-2422af1ade07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:59.513733 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c6ec4-7057-4221-b6ef-2422af1ade07 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:59.517068 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc74415-9a02-4a3d-a179-f019c265ac28 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:59.518458 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4fc74415-9a02-4a3d-a179-f019c265ac28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:59.523267 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:59.530412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1120/2246] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:39:59.553401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc74415-9a02-4a3d-a179-f019c265ac28 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:59.553401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4b9a8-8e02-49ee-a6b2-f6640aaa302b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:59.553401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4b9a8-8e02-49ee-a6b2-f6640aaa302b service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:59.553401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4b9a8-8e02-49ee-a6b2-f6640aaa302b service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:59.553829 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4b9a8-8e02-49ee-a6b2-f6640aaa302b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:59.553829 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b6d4b9a8-8e02-49ee-a6b2-f6640aaa302b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:39:59.557324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c6ec4-7057-4221-b6ef-2422af1ade07 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:39:59.598323 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:39:59.642597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4b9a8-8e02-49ee-a6b2-f6640aaa302b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:59.644332 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4b9a8-8e02-49ee-a6b2-f6640aaa302b service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:59.650350 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c6ec4-7057-4221-b6ef-2422af1ade07 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:39:59.655262 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:39:59.656114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:39:59.656494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:39:59.656951 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] PATH_INFO: `/v3/users/2027b8eab6e140b796a4d20a015704d7` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:39:59.657836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1127/2247] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:39:59 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:39:59.674935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2027b8eab6e140b796a4d20a015704d7)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:39:59.678263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:39:59.893681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2027b8eab6e140b796a4d20a015704d7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:59.894585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2027b8eab6e140b796a4d20a015704d7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:59.978815 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2027b8eab6e140b796a4d20a015704d7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:59.979712 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2027b8eab6e140b796a4d20a015704d7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:39:59.988836 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2027b8eab6e140b796a4d20a015704d7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:00.015838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1121/2248] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:39:59 2022] DELETE /identity/v3/users/2027b8eab6e140b796a4d20a015704d7 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:00.020757 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86835c-7d68-4d67-a8e6-ca0d4c5a9066 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:00.021771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86835c-7d68-4d67-a8e6-ca0d4c5a9066 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:00.021771 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86835c-7d68-4d67-a8e6-ca0d4c5a9066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:00.044872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-db86835c-7d68-4d67-a8e6-ca0d4c5a9066 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:00.046704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-db86835c-7d68-4d67-a8e6-ca0d4c5a9066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:00.134203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1128/2249] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:00.143304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:00.146513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:00.227135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71c384-3d90-4db1-8e6c-6bb455cce151 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:00.235858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d287b1-c7ac-46e0-8ff4-619426f866ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:00.236497 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d287b1-c7ac-46e0-8ff4-619426f866ae service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:00.236777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d287b1-c7ac-46e0-8ff4-619426f866ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:00.237042 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d287b1-c7ac-46e0-8ff4-619426f866ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:00.249370 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d287b1-c7ac-46e0-8ff4-619426f866ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:00.249370 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d287b1-c7ac-46e0-8ff4-619426f866ae service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:00.266296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1122/2250] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:00.781863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86835c-7d68-4d67-a8e6-ca0d4c5a9066 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:00.783684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-db86835c-7d68-4d67-a8e6-ca0d4c5a9066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:00.797886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86835c-7d68-4d67-a8e6-ca0d4c5a9066 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:00.807960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:00.808770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:00.809153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:00.809528 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] PATH_INFO: `/v3/projects/32aa73aeb0724ddbbe498f5ca95ff88e` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:00.810830 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32aa73aeb0724ddbbe498f5ca95ff88e)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:00.813178 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:00.943109 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32aa73aeb0724ddbbe498f5ca95ff88e', 'request_id': 'req-56870a06-f73f-4692-a316-2eb0399efa98'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:00.956567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32aa73aeb0724ddbbe498f5ca95ff88e', 'request_id': 'req-56870a06-f73f-4692-a316-2eb0399efa98'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:01.024174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32aa73aeb0724ddbbe498f5ca95ff88e', 'request_id': 'req-56870a06-f73f-4692-a316-2eb0399efa98'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:01.037493 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] The token cache is being invalidate because project 32aa73aeb0724ddbbe498f5ca95ff88e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:01.043900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:01.044710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-56870a06-f73f-4692-a316-2eb0399efa98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:01.046598 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1129/2251] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:00 2022] DELETE /identity/v3/projects/32aa73aeb0724ddbbe498f5ca95ff88e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:01.053501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dc574-107f-4552-bd13-7d2daad1269e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:01.054133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dc574-107f-4552-bd13-7d2daad1269e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:01.054133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dc574-107f-4552-bd13-7d2daad1269e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:01.083620 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c15dc574-107f-4552-bd13-7d2daad1269e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:01.084891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c15dc574-107f-4552-bd13-7d2daad1269e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:01.113634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:01.161741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1123/2252] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:01.172016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246afad8-69f4-464d-b254-cb6a4fbdba05 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:01.172317 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246afad8-69f4-464d-b254-cb6a4fbdba05 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:01.172596 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246afad8-69f4-464d-b254-cb6a4fbdba05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:01.192377 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-246afad8-69f4-464d-b254-cb6a4fbdba05 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:01.193675 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-246afad8-69f4-464d-b254-cb6a4fbdba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:01.201549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1130/2253] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:01.205834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dc574-107f-4552-bd13-7d2daad1269e None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:01.216333 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dc574-107f-4552-bd13-7d2daad1269e None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:01.221054 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2098c-3eeb-4718-a70a-2075de392f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:01.221660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2098c-3eeb-4718-a70a-2075de392f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:01.221940 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2098c-3eeb-4718-a70a-2075de392f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:01.222271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2098c-3eeb-4718-a70a-2075de392f72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:01.222941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2098c-3eeb-4718-a70a-2075de392f72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:40:01.223243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2098c-3eeb-4718-a70a-2075de392f72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:01.224852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2098c-3eeb-4718-a70a-2075de392f72 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:01.237576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1124/2254] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:40:01.241587 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246afad8-69f4-464d-b254-cb6a4fbdba05 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:01.251235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246afad8-69f4-464d-b254-cb6a4fbdba05 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:01.256147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae21786-c53e-400f-a87e-fb1e10883999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:01.256788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae21786-c53e-400f-a87e-fb1e10883999 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:01.257062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae21786-c53e-400f-a87e-fb1e10883999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:01.257869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae21786-c53e-400f-a87e-fb1e10883999 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:01.258796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae21786-c53e-400f-a87e-fb1e10883999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:01.260701 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae21786-c53e-400f-a87e-fb1e10883999 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:01.748463 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1131/2255] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:01 2022] POST /identity/v3/projects => generated 373 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:01.750019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2098c-3eeb-4718-a70a-2075de392f72 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:01.760941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2098c-3eeb-4718-a70a-2075de392f72 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:01.834900 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def97b16-6cd5-45c9-a25f-0a75914ce8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:01.835524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def97b16-6cd5-45c9-a25f-0a75914ce8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:01.835808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def97b16-6cd5-45c9-a25f-0a75914ce8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:01.836097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def97b16-6cd5-45c9-a25f-0a75914ce8d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:01.836799 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def97b16-6cd5-45c9-a25f-0a75914ce8d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:01.838803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def97b16-6cd5-45c9-a25f-0a75914ce8d2 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:01.954868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1125/2256] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:01 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:01.959987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae21786-c53e-400f-a87e-fb1e10883999 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:01.973320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae21786-c53e-400f-a87e-fb1e10883999 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:01.980527 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6353ea-7454-475a-a27c-1a387f287f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:01.981138 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6353ea-7454-475a-a27c-1a387f287f9b admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:01.981459 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6353ea-7454-475a-a27c-1a387f287f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:01.981825 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6353ea-7454-475a-a27c-1a387f287f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:01.982543 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6353ea-7454-475a-a27c-1a387f287f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:01.986476 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6353ea-7454-475a-a27c-1a387f287f9b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:02.003591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def97b16-6cd5-45c9-a25f-0a75914ce8d2 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:02.009387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1132/2257] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:02.017103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def97b16-6cd5-45c9-a25f-0a75914ce8d2 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:02.023016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f43480-0e53-40cc-ab0e-37ea2409935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:02.023631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f43480-0e53-40cc-ab0e-37ea2409935b admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:02.023912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f43480-0e53-40cc-ab0e-37ea2409935b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:02.024238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f43480-0e53-40cc-ab0e-37ea2409935b admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:02.024921 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f43480-0e53-40cc-ab0e-37ea2409935b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:02.026563 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f43480-0e53-40cc-ab0e-37ea2409935b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:02.043339 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1126/2258] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:02.044501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6353ea-7454-475a-a27c-1a387f287f9b admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:02.055739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6353ea-7454-475a-a27c-1a387f287f9b admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:02.060883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c9821-5ced-4bbf-be49-57f1e2dcb439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:02.061603 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c9821-5ced-4bbf-be49-57f1e2dcb439 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:02.061890 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c9821-5ced-4bbf-be49-57f1e2dcb439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:02.062269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c9821-5ced-4bbf-be49-57f1e2dcb439 admin admin] PATH_INFO: `/v3/projects/daeab86fb2b94f31b63b144d587f78b4/users/478da7d0ce9540cc95566689a69790c0/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:02.086886 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c9821-5ced-4bbf-be49-57f1e2dcb439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daeab86fb2b94f31b63b144d587f78b4, user_id=478da7d0ce9540cc95566689a69790c0, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:02.089360 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c9821-5ced-4bbf-be49-57f1e2dcb439 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:02.095942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:02.230643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1133/2259] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:02 2022] PUT /identity/v3/projects/daeab86fb2b94f31b63b144d587f78b4/users/478da7d0ce9540cc95566689a69790c0/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:02.243558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c3109-793f-4d35-a911-7f764b4e0e2b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:02.244197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c3109-793f-4d35-a911-7f764b4e0e2b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:02.244298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c3109-793f-4d35-a911-7f764b4e0e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:02.279035 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-4a2c3109-793f-4d35-a911-7f764b4e0e2b None None] MFA Rules not processed for user `478da7d0ce9540cc95566689a69790c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:02.280158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4a2c3109-793f-4d35-a911-7f764b4e0e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:02.371319 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1127/2260] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:02.377336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c9821-5ced-4bbf-be49-57f1e2dcb439 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:02.455192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c9821-5ced-4bbf-be49-57f1e2dcb439 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:02.461187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3bd47-5dfa-4d0d-9eab-59f317f8947e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:02.461962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3bd47-5dfa-4d0d-9eab-59f317f8947e admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:02.462290 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3bd47-5dfa-4d0d-9eab-59f317f8947e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:02.462631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3bd47-5dfa-4d0d-9eab-59f317f8947e admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:02.463405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3bd47-5dfa-4d0d-9eab-59f317f8947e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:02.465354 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3bd47-5dfa-4d0d-9eab-59f317f8947e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:02.561371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1134/2261] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:02 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:02.562648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c3109-793f-4d35-a911-7f764b4e0e2b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:02.576199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c3109-793f-4d35-a911-7f764b4e0e2b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:02.591513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:02.646530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d604021-2058-45e3-84fe-472192cfd294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:02.647365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d604021-2058-45e3-84fe-472192cfd294 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:02.648243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d604021-2058-45e3-84fe-472192cfd294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:02.648518 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d604021-2058-45e3-84fe-472192cfd294 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:02.649171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d604021-2058-45e3-84fe-472192cfd294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:02.651348 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d604021-2058-45e3-84fe-472192cfd294 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:02.742506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1128/2262] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:02 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:02.744020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3bd47-5dfa-4d0d-9eab-59f317f8947e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:02.758242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3bd47-5dfa-4d0d-9eab-59f317f8947e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:02.763544 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15c1ab-cb92-4ab8-8a85-0b4b6391229f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:02.764276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15c1ab-cb92-4ab8-8a85-0b4b6391229f admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:02.764561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15c1ab-cb92-4ab8-8a85-0b4b6391229f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:02.764833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15c1ab-cb92-4ab8-8a85-0b4b6391229f admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:02.765481 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15c1ab-cb92-4ab8-8a85-0b4b6391229f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:02.767171 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15c1ab-cb92-4ab8-8a85-0b4b6391229f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:02.779799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1135/2263] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:02.786613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d604021-2058-45e3-84fe-472192cfd294 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:02.797906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d604021-2058-45e3-84fe-472192cfd294 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:02.805257 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f05932-c7a6-4786-a2e4-c80a541a1acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:02.806090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f05932-c7a6-4786-a2e4-c80a541a1acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:02.806605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f05932-c7a6-4786-a2e4-c80a541a1acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:02.806864 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f05932-c7a6-4786-a2e4-c80a541a1acb admin admin] PATH_INFO: `/v3/projects/e714309b8e32405abf45fb17b0c951ca/users/df8243b3febf420fb39b276032085c92/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:02.827480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f05932-c7a6-4786-a2e4-c80a541a1acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e714309b8e32405abf45fb17b0c951ca, user_id=df8243b3febf420fb39b276032085c92, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:02.830332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f05932-c7a6-4786-a2e4-c80a541a1acb admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:02.890577 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1129/2264] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:02 2022] PUT /identity/v3/projects/e714309b8e32405abf45fb17b0c951ca/users/df8243b3febf420fb39b276032085c92/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:02.894686 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15c1ab-cb92-4ab8-8a85-0b4b6391229f admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:02.909365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15c1ab-cb92-4ab8-8a85-0b4b6391229f admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:02.979436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66355052-b092-4715-80f8-9460bddfbd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:02.980724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66355052-b092-4715-80f8-9460bddfbd5a admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:02.981011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66355052-b092-4715-80f8-9460bddfbd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:02.981284 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66355052-b092-4715-80f8-9460bddfbd5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:02.981936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66355052-b092-4715-80f8-9460bddfbd5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:02.983557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66355052-b092-4715-80f8-9460bddfbd5a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:02.994197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1136/2265] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:40:02.998457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f05932-c7a6-4786-a2e4-c80a541a1acb admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:03.011993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f05932-c7a6-4786-a2e4-c80a541a1acb admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:03.019601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf27055-384d-478d-9560-29beee38c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:03.020434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf27055-384d-478d-9560-29beee38c3ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:03.020785 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf27055-384d-478d-9560-29beee38c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:03.021118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf27055-384d-478d-9560-29beee38c3ad admin admin] PATH_INFO: `/v3/projects/e714309b8e32405abf45fb17b0c951ca/users/df8243b3febf420fb39b276032085c92/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:03.023775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf27055-384d-478d-9560-29beee38c3ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=e714309b8e32405abf45fb17b0c951ca, user_id=df8243b3febf420fb39b276032085c92, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:03.026548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf27055-384d-478d-9560-29beee38c3ad admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:03.038307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1130/2266] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:02 2022] PUT /identity/v3/projects/e714309b8e32405abf45fb17b0c951ca/users/df8243b3febf420fb39b276032085c92/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:03.054074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da798e-0b7f-48c3-a6b3-69a25434de6c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:03.054074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da798e-0b7f-48c3-a6b3-69a25434de6c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:03.054074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da798e-0b7f-48c3-a6b3-69a25434de6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:03.084825 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-83da798e-0b7f-48c3-a6b3-69a25434de6c None None] MFA Rules not processed for user `df8243b3febf420fb39b276032085c92`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:03.086567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-83da798e-0b7f-48c3-a6b3-69a25434de6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:03.158035 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:03.187928 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1137/2267] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:03.196039 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf27055-384d-478d-9560-29beee38c3ad admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:03.197971 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-faf27055-384d-478d-9560-29beee38c3ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:03.267510 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf27055-384d-478d-9560-29beee38c3ad admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:03.271334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e1ee7-be72-4953-80e9-d5dbd706a208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:03.272033 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e1ee7-be72-4953-80e9-d5dbd706a208 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:03.272371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e1ee7-be72-4953-80e9-d5dbd706a208 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:03.272739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e1ee7-be72-4953-80e9-d5dbd706a208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:03.355199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e1ee7-be72-4953-80e9-d5dbd706a208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:03.356660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e1ee7-be72-4953-80e9-d5dbd706a208 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:03.368039 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1131/2268] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:07.690131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da798e-0b7f-48c3-a6b3-69a25434de6c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:07.751218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da798e-0b7f-48c3-a6b3-69a25434de6c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:07.755681 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:07.756301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:07.756606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:07.756878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] PATH_INFO: `/v3/users/478da7d0ce9540cc95566689a69790c0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:07.757987 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=478da7d0ce9540cc95566689a69790c0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:07.760039 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:07.921994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478da7d0ce9540cc95566689a69790c0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:07.922673 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478da7d0ce9540cc95566689a69790c0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:07.929728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478da7d0ce9540cc95566689a69790c0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:08.011110 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478da7d0ce9540cc95566689a69790c0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:08.011714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478da7d0ce9540cc95566689a69790c0'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:08.031117 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1138/2269] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:07 2022] DELETE /identity/v3/users/478da7d0ce9540cc95566689a69790c0 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:08.035902 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24178-1303-45d7-9dd8-7ed63d02cfef None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:08.036380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24178-1303-45d7-9dd8-7ed63d02cfef None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:08.036480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e24178-1303-45d7-9dd8-7ed63d02cfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:08.056047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-34e24178-1303-45d7-9dd8-7ed63d02cfef None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:08.057223 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-34e24178-1303-45d7-9dd8-7ed63d02cfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:08.074869 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:08.117526 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1132/2270] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:08.127815 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:08.129197 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:08.189977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc2935-c0bb-4c10-ad01-c6b4f8ce5852 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:08.193522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a5c85-3786-458e-b983-5ee0b4915971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:08.194219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a5c85-3786-458e-b983-5ee0b4915971 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:08.194495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a5c85-3786-458e-b983-5ee0b4915971 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:08.194760 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a5c85-3786-458e-b983-5ee0b4915971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:08.204774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a5c85-3786-458e-b983-5ee0b4915971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:08.206285 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a5c85-3786-458e-b983-5ee0b4915971 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:08.216564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1139/2271] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:08.642119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e24178-1303-45d7-9dd8-7ed63d02cfef None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:08.647298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-34e24178-1303-45d7-9dd8-7ed63d02cfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:08.655586 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e24178-1303-45d7-9dd8-7ed63d02cfef None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:08.661298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:08.661946 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:08.662344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:08.662538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] PATH_INFO: `/v3/projects/daeab86fb2b94f31b63b144d587f78b4` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:08.663731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daeab86fb2b94f31b63b144d587f78b4)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:08.665783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:08.800949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daeab86fb2b94f31b63b144d587f78b4', 'request_id': 'req-aa4dee59-ff39-44e4-8231-b3160b1a5423'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:08.808775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daeab86fb2b94f31b63b144d587f78b4', 'request_id': 'req-aa4dee59-ff39-44e4-8231-b3160b1a5423'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:08.869313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daeab86fb2b94f31b63b144d587f78b4', 'request_id': 'req-aa4dee59-ff39-44e4-8231-b3160b1a5423'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:08.880285 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] The token cache is being invalidate because project daeab86fb2b94f31b63b144d587f78b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:08.880566 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:08.881183 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:08.882764 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1133/2272] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:08 2022] DELETE /identity/v3/projects/daeab86fb2b94f31b63b144d587f78b4 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:08.886720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a5c85-3786-458e-b983-5ee0b4915971 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:08.896299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a5c85-3786-458e-b983-5ee0b4915971 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:08.905474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:08.950786 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:08.951409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:08.951689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:08.951962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] PATH_INFO: `/v3/users/df8243b3febf420fb39b276032085c92` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:08.953060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=df8243b3febf420fb39b276032085c92)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:08.955018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:09.127674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df8243b3febf420fb39b276032085c92'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:09.128320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df8243b3febf420fb39b276032085c92'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:09.135132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df8243b3febf420fb39b276032085c92'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:09.212038 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df8243b3febf420fb39b276032085c92'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:09.212658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df8243b3febf420fb39b276032085c92'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:09.230723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1140/2273] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:08 2022] DELETE /identity/v3/users/df8243b3febf420fb39b276032085c92 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:40:09.750858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:09.752362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:09.769884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4dee59-ff39-44e4-8231-b3160b1a5423 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:09.829916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:09.830582 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:09.830870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:09.831185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] PATH_INFO: `/v3/projects/e714309b8e32405abf45fb17b0c951ca` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:09.832356 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e714309b8e32405abf45fb17b0c951ca)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:09.837266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:09.968580 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:09.969998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:09.971850 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:09.973197 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e714309b8e32405abf45fb17b0c951ca', 'request_id': 'req-81c6f292-1f9c-41a3-ae0f-551252088fec'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:09.979817 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e714309b8e32405abf45fb17b0c951ca', 'request_id': 'req-81c6f292-1f9c-41a3-ae0f-551252088fec'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:09.992435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387094db-83e1-45e6-a288-294cefe256ba admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:10.063191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e714309b8e32405abf45fb17b0c951ca', 'request_id': 'req-81c6f292-1f9c-41a3-ae0f-551252088fec'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:10.066849 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122525a-c35b-4194-a4c7-0f2fe017244a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:10.067600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122525a-c35b-4194-a4c7-0f2fe017244a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.071738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122525a-c35b-4194-a4c7-0f2fe017244a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.071738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122525a-c35b-4194-a4c7-0f2fe017244a admin admin] PATH_INFO: `/v3/users/66ba9c37867c4ece8662d34b4a642a41` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.071738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122525a-c35b-4194-a4c7-0f2fe017244a admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ba9c37867c4ece8662d34b4a642a41)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:10.072823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122525a-c35b-4194-a4c7-0f2fe017244a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:10.075704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] The token cache is being invalidate because project e714309b8e32405abf45fb17b0c951ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:10.076254 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:10.076906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-81c6f292-1f9c-41a3-ae0f-551252088fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:10.078513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1134/2274] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:09 2022] DELETE /identity/v3/projects/e714309b8e32405abf45fb17b0c951ca => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:10.084923 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ee47-f7b3-42c9-8c61-8cea555e89f5 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.085206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ee47-f7b3-42c9-8c61-8cea555e89f5 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.085560 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ee47-f7b3-42c9-8c61-8cea555e89f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.108431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-91a7ee47-f7b3-42c9-8c61-8cea555e89f5 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:10.109633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-91a7ee47-f7b3-42c9-8c61-8cea555e89f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:10.118406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1135/2275] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:10.126074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26888c-1c57-417a-934e-7aa8972e1fa0 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.126367 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26888c-1c57-417a-934e-7aa8972e1fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.126658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26888c-1c57-417a-934e-7aa8972e1fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.146217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-4a26888c-1c57-417a-934e-7aa8972e1fa0 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:10.147419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4a26888c-1c57-417a-934e-7aa8972e1fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:10.155121 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1136/2276] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:40:10.159118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26888c-1c57-417a-934e-7aa8972e1fa0 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:10.168052 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26888c-1c57-417a-934e-7aa8972e1fa0 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:10.172662 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d361a7e-375e-4c7e-b33e-b6a855378164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:10.173303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d361a7e-375e-4c7e-b33e-b6a855378164 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.173611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d361a7e-375e-4c7e-b33e-b6a855378164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.173885 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d361a7e-375e-4c7e-b33e-b6a855378164 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.174555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d361a7e-375e-4c7e-b33e-b6a855378164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:40:10.174854 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d361a7e-375e-4c7e-b33e-b6a855378164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:10.176764 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d361a7e-375e-4c7e-b33e-b6a855378164 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:10.189328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1137/2277] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:40:10.193529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d361a7e-375e-4c7e-b33e-b6a855378164 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:10.203670 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d361a7e-375e-4c7e-b33e-b6a855378164 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:10.208500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cd0cf-69d4-4990-8cd4-40422ebedfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:10.209097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cd0cf-69d4-4990-8cd4-40422ebedfa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.209370 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cd0cf-69d4-4990-8cd4-40422ebedfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.209638 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7cd0cf-69d4-4990-8cd4-40422ebedfa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.210387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cd0cf-69d4-4990-8cd4-40422ebedfa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:10.212328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7cd0cf-69d4-4990-8cd4-40422ebedfa3 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:10.239491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:10.249045 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b122525a-c35b-4194-a4c7-0f2fe017244a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ba9c37867c4ece8662d34b4a642a41'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:10.249684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b122525a-c35b-4194-a4c7-0f2fe017244a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ba9c37867c4ece8662d34b4a642a41'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:10.258467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b122525a-c35b-4194-a4c7-0f2fe017244a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ba9c37867c4ece8662d34b4a642a41'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:10.293427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1138/2278] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:10 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:40:10.297398 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cd0cf-69d4-4990-8cd4-40422ebedfa3 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:10.298970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1d7cd0cf-69d4-4990-8cd4-40422ebedfa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:10.331717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b122525a-c35b-4194-a4c7-0f2fe017244a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ba9c37867c4ece8662d34b4a642a41'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:10.332423 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b122525a-c35b-4194-a4c7-0f2fe017244a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ba9c37867c4ece8662d34b4a642a41'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:10.351470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1141/2279] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:09 2022] DELETE /identity/v3/users/66ba9c37867c4ece8662d34b4a642a41 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:40:10.356506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7401f00-386d-4760-af67-edb604ca1e68 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.356791 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7401f00-386d-4760-af67-edb604ca1e68 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.357061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7401f00-386d-4760-af67-edb604ca1e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.362532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7cd0cf-69d4-4990-8cd4-40422ebedfa3 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:10.376460 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-a7401f00-386d-4760-af67-edb604ca1e68 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:10.377740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a7401f00-386d-4760-af67-edb604ca1e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:10.419418 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06882179-633d-4458-a7ab-93da7b58cd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:10.419962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06882179-633d-4458-a7ab-93da7b58cd8d admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.420227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06882179-633d-4458-a7ab-93da7b58cd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.420501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06882179-633d-4458-a7ab-93da7b58cd8d admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.421147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06882179-633d-4458-a7ab-93da7b58cd8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:10.423132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06882179-633d-4458-a7ab-93da7b58cd8d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:10.436099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1142/2280] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:40:10.445877 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7401f00-386d-4760-af67-edb604ca1e68 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:10.447196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a7401f00-386d-4760-af67-edb604ca1e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:10.491910 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1139/2281] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:10 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:40:10.496359 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06882179-633d-4458-a7ab-93da7b58cd8d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:10.503722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7401f00-386d-4760-af67-edb604ca1e68 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:10.506324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06882179-633d-4458-a7ab-93da7b58cd8d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:10.507405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d2d0d-0fd8-42f3-8b80-a8783cae86e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:10.508000 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d2d0d-0fd8-42f3-8b80-a8783cae86e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.508305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d2d0d-0fd8-42f3-8b80-a8783cae86e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.508575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d2d0d-0fd8-42f3-8b80-a8783cae86e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.511316 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ca7da-9fa8-49fe-861d-4d27d0315030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:10.511993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ca7da-9fa8-49fe-861d-4d27d0315030 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.517249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ca7da-9fa8-49fe-861d-4d27d0315030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.517249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ca7da-9fa8-49fe-861d-4d27d0315030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.517249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ca7da-9fa8-49fe-861d-4d27d0315030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:10.517249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ca7da-9fa8-49fe-861d-4d27d0315030 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:10.517685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d2d0d-0fd8-42f3-8b80-a8783cae86e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:10.519214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d2d0d-0fd8-42f3-8b80-a8783cae86e7 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:10.525303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1140/2282] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:40:10.528849 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1143/2283] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:40:10.529124 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ca7da-9fa8-49fe-861d-4d27d0315030 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:10.538787 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ca7da-9fa8-49fe-861d-4d27d0315030 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:10.544504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109804c-ffbb-4569-a22e-e3716ba83f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:10.545101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109804c-ffbb-4569-a22e-e3716ba83f0c admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.545407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109804c-ffbb-4569-a22e-e3716ba83f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.545675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109804c-ffbb-4569-a22e-e3716ba83f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.546332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109804c-ffbb-4569-a22e-e3716ba83f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:10.547953 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109804c-ffbb-4569-a22e-e3716ba83f0c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:10.559261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1141/2284] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:40:10.563373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d2d0d-0fd8-42f3-8b80-a8783cae86e7 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:10.573246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d2d0d-0fd8-42f3-8b80-a8783cae86e7 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:10.577894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86658f84-9cf9-415e-a0db-ec5104be5056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:10.578567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86658f84-9cf9-415e-a0db-ec5104be5056 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.578845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86658f84-9cf9-415e-a0db-ec5104be5056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.579114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86658f84-9cf9-415e-a0db-ec5104be5056 admin admin] PATH_INFO: `/v3/projects/80abbd8aca9646e7ad9fba1e79900613/users/1c3808cb27af418ebe5137d2c439ac97/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.594889 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86658f84-9cf9-415e-a0db-ec5104be5056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80abbd8aca9646e7ad9fba1e79900613, user_id=1c3808cb27af418ebe5137d2c439ac97, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:10.597524 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86658f84-9cf9-415e-a0db-ec5104be5056 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:10.663656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1144/2285] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:10 2022] PUT /identity/v3/projects/80abbd8aca9646e7ad9fba1e79900613/users/1c3808cb27af418ebe5137d2c439ac97/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:10.674792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af95d33-9b4e-4ec7-8b43-83051d7a7eec None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.675000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af95d33-9b4e-4ec7-8b43-83051d7a7eec None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.675267 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af95d33-9b4e-4ec7-8b43-83051d7a7eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.682892 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:10.705970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-6af95d33-9b4e-4ec7-8b43-83051d7a7eec None None] MFA Rules not processed for user `1c3808cb27af418ebe5137d2c439ac97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:10.708297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6af95d33-9b4e-4ec7-8b43-83051d7a7eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:10.801202 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1142/2286] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:10.813229 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86658f84-9cf9-415e-a0db-ec5104be5056 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:10.815736 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-86658f84-9cf9-415e-a0db-ec5104be5056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:10.857328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:10.884403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86658f84-9cf9-415e-a0db-ec5104be5056 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:10.888068 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce09353-807b-49ae-b816-501231532aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:10.888679 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09353-807b-49ae-b816-501231532aab service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:10.888961 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09353-807b-49ae-b816-501231532aab service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:10.889254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce09353-807b-49ae-b816-501231532aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:10.897666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce09353-807b-49ae-b816-501231532aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:10.899659 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce09353-807b-49ae-b816-501231532aab service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:10.908503 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1145/2287] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:11.064852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af95d33-9b4e-4ec7-8b43-83051d7a7eec None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:11.067860 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6af95d33-9b4e-4ec7-8b43-83051d7a7eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:11.139119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af95d33-9b4e-4ec7-8b43-83051d7a7eec None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:11.143721 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:11.144346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:11.144656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:11.144928 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] PATH_INFO: `/v3/projects/92d8ab995dac41d09c33a5e2f0c668dd` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:11.146059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92d8ab995dac41d09c33a5e2f0c668dd)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:11.150623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:11.243895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92d8ab995dac41d09c33a5e2f0c668dd', 'request_id': 'req-af918e25-9ad8-44ab-a1b6-a97badf617d6'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:11.250702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92d8ab995dac41d09c33a5e2f0c668dd', 'request_id': 'req-af918e25-9ad8-44ab-a1b6-a97badf617d6'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:11.325729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92d8ab995dac41d09c33a5e2f0c668dd', 'request_id': 'req-af918e25-9ad8-44ab-a1b6-a97badf617d6'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:11.339432 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] The token cache is being invalidate because project 92d8ab995dac41d09c33a5e2f0c668dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:11.339715 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:11.340347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:11.341935 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1143/2288] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:11 2022] DELETE /identity/v3/projects/92d8ab995dac41d09c33a5e2f0c668dd => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:11.346137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce09353-807b-49ae-b816-501231532aab service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:11.357863 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce09353-807b-49ae-b816-501231532aab service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:11.413264 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a2354-0d12-47dc-97b5-2f32fce2cdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:11.414010 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a2354-0d12-47dc-97b5-2f32fce2cdb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:11.414322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a2354-0d12-47dc-97b5-2f32fce2cdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:11.414626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a2354-0d12-47dc-97b5-2f32fce2cdb6 admin admin] PATH_INFO: `/v3/users/bec2f6b29ac6413f9316fa35dc40bba5` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:11.415815 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a2354-0d12-47dc-97b5-2f32fce2cdb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec2f6b29ac6413f9316fa35dc40bba5)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:11.417738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a2354-0d12-47dc-97b5-2f32fce2cdb6 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:11.643366 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b13a2354-0d12-47dc-97b5-2f32fce2cdb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec2f6b29ac6413f9316fa35dc40bba5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:11.644028 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b13a2354-0d12-47dc-97b5-2f32fce2cdb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec2f6b29ac6413f9316fa35dc40bba5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:11.652832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b13a2354-0d12-47dc-97b5-2f32fce2cdb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec2f6b29ac6413f9316fa35dc40bba5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:11.760088 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b13a2354-0d12-47dc-97b5-2f32fce2cdb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec2f6b29ac6413f9316fa35dc40bba5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:11.760830 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b13a2354-0d12-47dc-97b5-2f32fce2cdb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec2f6b29ac6413f9316fa35dc40bba5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:11.780670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1146/2289] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:11 2022] DELETE /identity/v3/users/bec2f6b29ac6413f9316fa35dc40bba5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:40:12.173906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:12.177223 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:12.188989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af918e25-9ad8-44ab-a1b6-a97badf617d6 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:12.209088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:12.249067 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:12.249594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.249875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.250193 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] PATH_INFO: `/v3/projects/18273080be944c70957d8ea9d96cb359` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.251643 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=18273080be944c70957d8ea9d96cb359)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:12.253562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:12.347205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8bcfd-4465-4a49-81ff-4d450d11d33e None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.347823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8bcfd-4465-4a49-81ff-4d450d11d33e None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.347823 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8bcfd-4465-4a49-81ff-4d450d11d33e None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.349350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1147/2290] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:40:12.355755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c695b36-e63a-4827-bc1d-fa12bc0e8887 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.356576 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c695b36-e63a-4827-bc1d-fa12bc0e8887 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.357026 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c695b36-e63a-4827-bc1d-fa12bc0e8887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.360153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18273080be944c70957d8ea9d96cb359', 'request_id': 'req-b4c83131-b97c-4671-943d-d8e479f143bf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:12.367021 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18273080be944c70957d8ea9d96cb359', 'request_id': 'req-b4c83131-b97c-4671-943d-d8e479f143bf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:12.378832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2c695b36-e63a-4827-bc1d-fa12bc0e8887 None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:12.380867 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2c695b36-e63a-4827-bc1d-fa12bc0e8887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:12.430203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:12.449495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1148/2291] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:40:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:40:12.451623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18273080be944c70957d8ea9d96cb359', 'request_id': 'req-b4c83131-b97c-4671-943d-d8e479f143bf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:12.462064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c695b36-e63a-4827-bc1d-fa12bc0e8887 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:12.463395 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2c695b36-e63a-4827-bc1d-fa12bc0e8887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:12.464861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] The token cache is being invalidate because project 18273080be944c70957d8ea9d96cb359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:12.465157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:12.465794 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4c83131-b97c-4671-943d-d8e479f143bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:12.467358 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1144/2292] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:12 2022] DELETE /identity/v3/projects/18273080be944c70957d8ea9d96cb359 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:12.472009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1f2f5-826e-4472-a355-ae7584faa4bf None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.472303 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1f2f5-826e-4472-a355-ae7584faa4bf None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.472580 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1f2f5-826e-4472-a355-ae7584faa4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.492808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-79c1f2f5-826e-4472-a355-ae7584faa4bf None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:12.493998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-79c1f2f5-826e-4472-a355-ae7584faa4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:12.523065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c695b36-e63a-4827-bc1d-fa12bc0e8887 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:12.527977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19c6b2-a817-4f6b-a2ee-07d35b12057f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286cfcbb8334421f87c0d426aa52a30a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:12.528729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19c6b2-a817-4f6b-a2ee-07d35b12057f service glance] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.529029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19c6b2-a817-4f6b-a2ee-07d35b12057f service glance] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.529293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19c6b2-a817-4f6b-a2ee-07d35b12057f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.531041 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ea19c6b2-a817-4f6b-a2ee-07d35b12057f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:12.544320 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19c6b2-a817-4f6b-a2ee-07d35b12057f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:12.545728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19c6b2-a817-4f6b-a2ee-07d35b12057f service glance] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:12.554245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1145/2293] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:40:12.556534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1149/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:40:12.563162 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f723f2-e7f0-421f-83af-53ca40f560ad None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.563445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f723f2-e7f0-421f-83af-53ca40f560ad None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.563716 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f723f2-e7f0-421f-83af-53ca40f560ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.584591 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-a0f723f2-e7f0-421f-83af-53ca40f560ad None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:12.586031 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a0f723f2-e7f0-421f-83af-53ca40f560ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:12.594558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1146/2295] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:12.598482 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19c6b2-a817-4f6b-a2ee-07d35b12057f service glance] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:12.609237 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19c6b2-a817-4f6b-a2ee-07d35b12057f service glance] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:12.613784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b8b62-2573-48d2-b4d3-09e4d92efdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:12.614411 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b8b62-2573-48d2-b4d3-09e4d92efdec admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.614686 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b8b62-2573-48d2-b4d3-09e4d92efdec admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.614952 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b8b62-2573-48d2-b4d3-09e4d92efdec admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.615588 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b8b62-2573-48d2-b4d3-09e4d92efdec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:40:12.615884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b8b62-2573-48d2-b4d3-09e4d92efdec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:12.621605 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b8b62-2573-48d2-b4d3-09e4d92efdec admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:12.630182 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1150/2296] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:40:12.634807 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f723f2-e7f0-421f-83af-53ca40f560ad None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:12.644043 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f723f2-e7f0-421f-83af-53ca40f560ad None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:12.650652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83fed0-9a8f-4c34-a72a-aeed841c9a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:12.650652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83fed0-9a8f-4c34-a72a-aeed841c9a50 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.650652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83fed0-9a8f-4c34-a72a-aeed841c9a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.650652 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83fed0-9a8f-4c34-a72a-aeed841c9a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.651248 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83fed0-9a8f-4c34-a72a-aeed841c9a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:12.652639 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83fed0-9a8f-4c34-a72a-aeed841c9a50 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:12.790632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153109a1-49b0-47b9-90fe-d0cceba8f2aa None None] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.790914 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153109a1-49b0-47b9-90fe-d0cceba8f2aa None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.791208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153109a1-49b0-47b9-90fe-d0cceba8f2aa None None] PATH_INFO: `/` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.792649 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1151/2297] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:40:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 19 02:40:12.797468 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b156dbe-d235-45ec-aa68-81ca0e6b3987 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.797747 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b156dbe-d235-45ec-aa68-81ca0e6b3987 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.798015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b156dbe-d235-45ec-aa68-81ca0e6b3987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.819082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-8b156dbe-d235-45ec-aa68-81ca0e6b3987 None None] MFA Rules not processed for user `b5ff8583997843258e315413abac172f`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:12.823684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8b156dbe-d235-45ec-aa68-81ca0e6b3987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:12.823821 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1147/2298] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:12 2022] POST /identity/v3/projects => generated 381 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:12.825984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83fed0-9a8f-4c34-a72a-aeed841c9a50 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:12.856368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:12.878872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1152/2299] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:40:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:40:12.890774 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b156dbe-d235-45ec-aa68-81ca0e6b3987 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:12.892525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8b156dbe-d235-45ec-aa68-81ca0e6b3987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:12.893616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83fed0-9a8f-4c34-a72a-aeed841c9a50 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:12.898379 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d9e99-97a0-4010-98e5-086fe2f6d12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:12.899506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9e99-97a0-4010-98e5-086fe2f6d12a admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.899779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9e99-97a0-4010-98e5-086fe2f6d12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.900047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d9e99-97a0-4010-98e5-086fe2f6d12a admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.900681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d9e99-97a0-4010-98e5-086fe2f6d12a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:12.902622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d9e99-97a0-4010-98e5-086fe2f6d12a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:12.903554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b156dbe-d235-45ec-aa68-81ca0e6b3987 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:12.907806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea95b3-a643-47a7-b5f3-8f7f8fa64fd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:12.908428 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea95b3-a643-47a7-b5f3-8f7f8fa64fd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:12.908703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea95b3-a643-47a7-b5f3-8f7f8fa64fd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:12.908969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea95b3-a643-47a7-b5f3-8f7f8fa64fd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:12.918333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea95b3-a643-47a7-b5f3-8f7f8fa64fd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:12.920351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea95b3-a643-47a7-b5f3-8f7f8fa64fd3 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:12.928372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1153/2300] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:40:12.992364 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1148/2301] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:12 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:40:13.002017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea95b3-a643-47a7-b5f3-8f7f8fa64fd3 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:13.014767 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2c0ce-f68f-4991-8950-6778d304a3da None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.019375 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea95b3-a643-47a7-b5f3-8f7f8fa64fd3 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:13.019508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2c0ce-f68f-4991-8950-6778d304a3da None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.019508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2c0ce-f68f-4991-8950-6778d304a3da None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.019508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1149/2302] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:40:13.022604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fa3d4-8f40-4e29-a7a6-bf40d81ffcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:13.023621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fa3d4-8f40-4e29-a7a6-bf40d81ffcd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.024444 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fa3d4-8f40-4e29-a7a6-bf40d81ffcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.024714 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fa3d4-8f40-4e29-a7a6-bf40d81ffcd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.025479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fa3d4-8f40-4e29-a7a6-bf40d81ffcd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:13.028177 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fa3d4-8f40-4e29-a7a6-bf40d81ffcd2 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:13.028920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366d425-0fc1-4a64-ba61-723c13729003 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.029292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366d425-0fc1-4a64-ba61-723c13729003 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.029749 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366d425-0fc1-4a64-ba61-723c13729003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.039269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1154/2303] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:13.043188 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fa3d4-8f40-4e29-a7a6-bf40d81ffcd2 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:13.050219 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b366d425-0fc1-4a64-ba61-723c13729003 None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:13.051404 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b366d425-0fc1-4a64-ba61-723c13729003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:13.053049 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fa3d4-8f40-4e29-a7a6-bf40d81ffcd2 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:13.057953 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152434c-f6bf-4d1b-a780-942ccfc9a57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:13.058509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152434c-f6bf-4d1b-a780-942ccfc9a57c admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.058783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152434c-f6bf-4d1b-a780-942ccfc9a57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.059066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152434c-f6bf-4d1b-a780-942ccfc9a57c admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.059674 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152434c-f6bf-4d1b-a780-942ccfc9a57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:13.061349 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152434c-f6bf-4d1b-a780-942ccfc9a57c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:13.071173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1155/2304] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:40:13.075374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152434c-f6bf-4d1b-a780-942ccfc9a57c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:13.085140 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152434c-f6bf-4d1b-a780-942ccfc9a57c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:13.090293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f796cac-8e84-4208-8fe1-92e05c7438b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:13.090918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796cac-8e84-4208-8fe1-92e05c7438b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.091191 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796cac-8e84-4208-8fe1-92e05c7438b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.091457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796cac-8e84-4208-8fe1-92e05c7438b9 admin admin] PATH_INFO: `/v3/projects/fffe605e616640949c15ce4becc31ffa/users/26ccd7e2170e4727b57356eb0cbff81d/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.108757 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f796cac-8e84-4208-8fe1-92e05c7438b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fffe605e616640949c15ce4becc31ffa, user_id=26ccd7e2170e4727b57356eb0cbff81d, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:13.111627 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f796cac-8e84-4208-8fe1-92e05c7438b9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:13.127926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1150/2305] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:40:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:13.142833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366d425-0fc1-4a64-ba61-723c13729003 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:13.153207 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366d425-0fc1-4a64-ba61-723c13729003 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:13.157371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5cd2f-3c55-496b-b5a7-422e9e385e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:13.157967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5cd2f-3c55-496b-b5a7-422e9e385e04 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.158301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5cd2f-3c55-496b-b5a7-422e9e385e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.158620 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5cd2f-3c55-496b-b5a7-422e9e385e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.169688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5cd2f-3c55-496b-b5a7-422e9e385e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:13.171855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a5cd2f-3c55-496b-b5a7-422e9e385e04 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:13.182007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1151/2306] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:13.194474 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1156/2307] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:13 2022] PUT /identity/v3/projects/fffe605e616640949c15ce4becc31ffa/users/26ccd7e2170e4727b57356eb0cbff81d/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:40:13.205412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72927bc-3f4b-4625-ac40-88dc9564b8d2 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.205691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72927bc-3f4b-4625-ac40-88dc9564b8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.205964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72927bc-3f4b-4625-ac40-88dc9564b8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.238404 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b72927bc-3f4b-4625-ac40-88dc9564b8d2 None None] MFA Rules not processed for user `26ccd7e2170e4727b57356eb0cbff81d`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:13.239597 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b72927bc-3f4b-4625-ac40-88dc9564b8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:13.306934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1152/2308] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:13.311604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f796cac-8e84-4208-8fe1-92e05c7438b9 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:13.340651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:13.376974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f796cac-8e84-4208-8fe1-92e05c7438b9 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:13.382121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06e838-994b-4bf6-8ded-12549bd4065c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:13.382738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06e838-994b-4bf6-8ded-12549bd4065c admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.383015 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06e838-994b-4bf6-8ded-12549bd4065c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.383989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06e838-994b-4bf6-8ded-12549bd4065c admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.384632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06e838-994b-4bf6-8ded-12549bd4065c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:13.386638 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06e838-994b-4bf6-8ded-12549bd4065c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:13.465039 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1157/2309] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:13 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:13.468941 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72927bc-3f4b-4625-ac40-88dc9564b8d2 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:13.479435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72927bc-3f4b-4625-ac40-88dc9564b8d2 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:13.499777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:13.542073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa4c6e-94a8-46dc-bfa1-1bc6329abbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:13.542707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa4c6e-94a8-46dc-bfa1-1bc6329abbf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.542992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa4c6e-94a8-46dc-bfa1-1bc6329abbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.543286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa4c6e-94a8-46dc-bfa1-1bc6329abbf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.543922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa4c6e-94a8-46dc-bfa1-1bc6329abbf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:13.545853 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa4c6e-94a8-46dc-bfa1-1bc6329abbf5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:13.661058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1153/2310] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:13 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:13.662477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06e838-994b-4bf6-8ded-12549bd4065c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:13.674503 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06e838-994b-4bf6-8ded-12549bd4065c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:13.681099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c11c7d-19b1-4529-a1e1-f231cc77a609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:13.681920 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c11c7d-19b1-4529-a1e1-f231cc77a609 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.682408 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c11c7d-19b1-4529-a1e1-f231cc77a609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.682684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c11c7d-19b1-4529-a1e1-f231cc77a609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.683558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c11c7d-19b1-4529-a1e1-f231cc77a609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:13.685718 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c11c7d-19b1-4529-a1e1-f231cc77a609 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:13.698906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1158/2311] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:13.703276 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa4c6e-94a8-46dc-bfa1-1bc6329abbf5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:13.719637 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa4c6e-94a8-46dc-bfa1-1bc6329abbf5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:13.724742 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20e075-ebc9-473f-9403-e3e04eece50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:13.725413 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20e075-ebc9-473f-9403-e3e04eece50b admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.725691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20e075-ebc9-473f-9403-e3e04eece50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.725990 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20e075-ebc9-473f-9403-e3e04eece50b admin admin] PATH_INFO: `/v3/projects/49110aa9ae454e449fd3020a17943b9a/users/3a3d6197785f440b903a66e869068ac1/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.745122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20e075-ebc9-473f-9403-e3e04eece50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=49110aa9ae454e449fd3020a17943b9a, user_id=3a3d6197785f440b903a66e869068ac1, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:13.747835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20e075-ebc9-473f-9403-e3e04eece50b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:13.859325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1154/2312] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:13 2022] PUT /identity/v3/projects/49110aa9ae454e449fd3020a17943b9a/users/3a3d6197785f440b903a66e869068ac1/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:13.863387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c11c7d-19b1-4529-a1e1-f231cc77a609 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:13.873340 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c11c7d-19b1-4529-a1e1-f231cc77a609 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:13.932927 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e3f63-ad84-40f3-bb54-bed7ecd2364f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:13.933470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e3f63-ad84-40f3-bb54-bed7ecd2364f admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.933790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e3f63-ad84-40f3-bb54-bed7ecd2364f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.934242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e3f63-ad84-40f3-bb54-bed7ecd2364f admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.935014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e3f63-ad84-40f3-bb54-bed7ecd2364f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:13.936628 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e3f63-ad84-40f3-bb54-bed7ecd2364f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:13.947375 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1159/2313] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:13.951624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20e075-ebc9-473f-9403-e3e04eece50b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:13.962632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20e075-ebc9-473f-9403-e3e04eece50b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:13.967558 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda41ff3-9902-474b-bc8f-848ec6832e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:13.968204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda41ff3-9902-474b-bc8f-848ec6832e00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.968479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda41ff3-9902-474b-bc8f-848ec6832e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.968771 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda41ff3-9902-474b-bc8f-848ec6832e00 admin admin] PATH_INFO: `/v3/projects/49110aa9ae454e449fd3020a17943b9a/users/3a3d6197785f440b903a66e869068ac1/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:13.970750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda41ff3-9902-474b-bc8f-848ec6832e00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49110aa9ae454e449fd3020a17943b9a, user_id=3a3d6197785f440b903a66e869068ac1, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:13.973374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda41ff3-9902-474b-bc8f-848ec6832e00 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:13.984307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1155/2314] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:13 2022] PUT /identity/v3/projects/49110aa9ae454e449fd3020a17943b9a/users/3a3d6197785f440b903a66e869068ac1/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:13.994539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5c88b-c1a4-41a8-a1ba-9e0b972d7975 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:13.999195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5c88b-c1a4-41a8-a1ba-9e0b972d7975 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:13.999195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5c88b-c1a4-41a8-a1ba-9e0b972d7975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:14.023567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-65d5c88b-c1a4-41a8-a1ba-9e0b972d7975 None None] MFA Rules not processed for user `3a3d6197785f440b903a66e869068ac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:14.024819 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-65d5c88b-c1a4-41a8-a1ba-9e0b972d7975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:14.096536 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1160/2315] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:14.108597 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda41ff3-9902-474b-bc8f-848ec6832e00 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:14.110325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-bda41ff3-9902-474b-bc8f-848ec6832e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:14.170365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda41ff3-9902-474b-bc8f-848ec6832e00 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:14.174180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a518b52-8a3a-4a6f-991f-f2a6fdd49191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:14.174777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a518b52-8a3a-4a6f-991f-f2a6fdd49191 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:14.174984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a518b52-8a3a-4a6f-991f-f2a6fdd49191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:14.175268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a518b52-8a3a-4a6f-991f-f2a6fdd49191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:14.186682 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a518b52-8a3a-4a6f-991f-f2a6fdd49191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:14.188052 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a518b52-8a3a-4a6f-991f-f2a6fdd49191 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:14.198082 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1156/2316] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:14.736789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5c88b-c1a4-41a8-a1ba-9e0b972d7975 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:14.761530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:14.814562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5c88b-c1a4-41a8-a1ba-9e0b972d7975 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:14.820031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:14.821575 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:14.822194 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:14.822521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] PATH_INFO: `/v3/users/26ccd7e2170e4727b57356eb0cbff81d` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:14.823935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ccd7e2170e4727b57356eb0cbff81d)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:14.826571 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:14.962894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ccd7e2170e4727b57356eb0cbff81d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:14.963531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ccd7e2170e4727b57356eb0cbff81d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:14.971021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ccd7e2170e4727b57356eb0cbff81d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:15.028074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ccd7e2170e4727b57356eb0cbff81d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:15.028698 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ccd7e2170e4727b57356eb0cbff81d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:15.047016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1161/2317] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:14 2022] DELETE /identity/v3/users/26ccd7e2170e4727b57356eb0cbff81d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:15.051549 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2448b7c-9548-4ce7-b535-40e4a01c496d None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:15.051861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2448b7c-9548-4ce7-b535-40e4a01c496d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:15.052157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2448b7c-9548-4ce7-b535-40e4a01c496d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:15.071313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-d2448b7c-9548-4ce7-b535-40e4a01c496d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:15.072498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d2448b7c-9548-4ce7-b535-40e4a01c496d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:15.135306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1157/2318] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:15.145418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:15.147251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:15.220321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b2479-23c7-4c27-a2a1-9f59c3fbfa68 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:15.224040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8bcc9c-6ae3-4492-a772-638c6ee85b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:15.224590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8bcc9c-6ae3-4492-a772-638c6ee85b5b service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:15.224874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8bcc9c-6ae3-4492-a772-638c6ee85b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:15.225145 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8bcc9c-6ae3-4492-a772-638c6ee85b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:15.234732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8bcc9c-6ae3-4492-a772-638c6ee85b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:15.236221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8bcc9c-6ae3-4492-a772-638c6ee85b5b service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:15.246208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1162/2319] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:15.619347 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2448b7c-9548-4ce7-b535-40e4a01c496d None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:15.620838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d2448b7c-9548-4ce7-b535-40e4a01c496d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:15.633951 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2448b7c-9548-4ce7-b535-40e4a01c496d None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:15.639206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:15.640329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:15.640453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:15.640921 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] PATH_INFO: `/v3/projects/fffe605e616640949c15ce4becc31ffa` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:15.642368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fffe605e616640949c15ce4becc31ffa)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:15.644550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:15.656875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:15.785217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fffe605e616640949c15ce4becc31ffa', 'request_id': 'req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:15.792786 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fffe605e616640949c15ce4becc31ffa', 'request_id': 'req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:15.965553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fffe605e616640949c15ce4becc31ffa', 'request_id': 'req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:15.977523 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] The token cache is being invalidate because project fffe605e616640949c15ce4becc31ffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:15.977804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:15.978461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:15.979924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1158/2320] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:15 2022] DELETE /identity/v3/projects/fffe605e616640949c15ce4becc31ffa => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:40:15.984746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8bcc9c-6ae3-4492-a772-638c6ee85b5b service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:15.995538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8bcc9c-6ae3-4492-a772-638c6ee85b5b service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:16.035341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:16.053099 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bb99d-17da-43d0-886a-282f25691e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:16.053724 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bb99d-17da-43d0-886a-282f25691e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:16.054005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bb99d-17da-43d0-886a-282f25691e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:16.054315 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bb99d-17da-43d0-886a-282f25691e05 admin admin] PATH_INFO: `/v3/users/3a3d6197785f440b903a66e869068ac1` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:16.055570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bb99d-17da-43d0-886a-282f25691e05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3d6197785f440b903a66e869068ac1)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:16.057504 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9bb99d-17da-43d0-886a-282f25691e05 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:16.516856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5f9bb99d-17da-43d0-886a-282f25691e05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3d6197785f440b903a66e869068ac1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:16.517500 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5f9bb99d-17da-43d0-886a-282f25691e05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3d6197785f440b903a66e869068ac1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:16.525073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5f9bb99d-17da-43d0-886a-282f25691e05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3d6197785f440b903a66e869068ac1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:16.652322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5f9bb99d-17da-43d0-886a-282f25691e05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3d6197785f440b903a66e869068ac1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:16.652985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5f9bb99d-17da-43d0-886a-282f25691e05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3d6197785f440b903a66e869068ac1'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:16.672552 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1163/2321] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:15 2022] DELETE /identity/v3/users/3a3d6197785f440b903a66e869068ac1 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:17.129231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:17.129879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:17.143366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2439e9-50c3-4cf1-8ae7-423d1ce17880 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:17.214667 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:17.216555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:17.216555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:17.216555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] PATH_INFO: `/v3/projects/49110aa9ae454e449fd3020a17943b9a` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:17.217811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49110aa9ae454e449fd3020a17943b9a)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:17.219776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:17.325043 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49110aa9ae454e449fd3020a17943b9a', 'request_id': 'req-f68a6f32-6da6-4610-ba8b-cdb2647cc685'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:17.331636 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49110aa9ae454e449fd3020a17943b9a', 'request_id': 'req-f68a6f32-6da6-4610-ba8b-cdb2647cc685'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:17.394282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49110aa9ae454e449fd3020a17943b9a', 'request_id': 'req-f68a6f32-6da6-4610-ba8b-cdb2647cc685'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:17.405833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] The token cache is being invalidate because project 49110aa9ae454e449fd3020a17943b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:17.406095 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:17.406950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f68a6f32-6da6-4610-ba8b-cdb2647cc685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:17.411219 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1159/2322] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:17 2022] DELETE /identity/v3/projects/49110aa9ae454e449fd3020a17943b9a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:17.415478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ff5fa-f641-4cac-ad4d-a16533f166a1 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:17.415972 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ff5fa-f641-4cac-ad4d-a16533f166a1 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:17.416072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ff5fa-f641-4cac-ad4d-a16533f166a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:17.438184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9f6ff5fa-f641-4cac-ad4d-a16533f166a1 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:17.439392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9f6ff5fa-f641-4cac-ad4d-a16533f166a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:17.506359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1164/2323] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:17.514726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dc3ce-e2a7-4053-a732-5f1be6b8fddf None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:17.515040 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dc3ce-e2a7-4053-a732-5f1be6b8fddf None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:17.515322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dc3ce-e2a7-4053-a732-5f1be6b8fddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:17.535653 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-8a3dc3ce-e2a7-4053-a732-5f1be6b8fddf None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:17.536859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8a3dc3ce-e2a7-4053-a732-5f1be6b8fddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:17.544949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1160/2324] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:17.549515 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ff5fa-f641-4cac-ad4d-a16533f166a1 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:17.561502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ff5fa-f641-4cac-ad4d-a16533f166a1 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:17.568011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eed46f-26cf-411f-a55b-a197ac804dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:17.568011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eed46f-26cf-411f-a55b-a197ac804dee admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:17.568011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eed46f-26cf-411f-a55b-a197ac804dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:17.568011 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eed46f-26cf-411f-a55b-a197ac804dee admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:17.568485 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eed46f-26cf-411f-a55b-a197ac804dee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:40:17.568926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eed46f-26cf-411f-a55b-a197ac804dee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:17.571381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eed46f-26cf-411f-a55b-a197ac804dee admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:17.588455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1165/2325] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:40:17.590251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dc3ce-e2a7-4053-a732-5f1be6b8fddf None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:17.610951 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dc3ce-e2a7-4053-a732-5f1be6b8fddf None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:17.616651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265cb5a-65db-45de-be59-47533c78a6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:17.619809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265cb5a-65db-45de-be59-47533c78a6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:17.619809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265cb5a-65db-45de-be59-47533c78a6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:17.619809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265cb5a-65db-45de-be59-47533c78a6b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:17.619809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265cb5a-65db-45de-be59-47533c78a6b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:17.620501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265cb5a-65db-45de-be59-47533c78a6b6 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:17.733856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1161/2326] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:17 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:17.738023 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eed46f-26cf-411f-a55b-a197ac804dee admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:17.747954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eed46f-26cf-411f-a55b-a197ac804dee admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:17.795700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:17.807919 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0daaf7-7659-4853-b437-5393cf8cac5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:17.808561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0daaf7-7659-4853-b437-5393cf8cac5a admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:17.808838 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0daaf7-7659-4853-b437-5393cf8cac5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:17.809105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0daaf7-7659-4853-b437-5393cf8cac5a admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:17.809731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0daaf7-7659-4853-b437-5393cf8cac5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:17.811822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0daaf7-7659-4853-b437-5393cf8cac5a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:17.877616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1166/2327] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:17 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:40:17.878823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265cb5a-65db-45de-be59-47533c78a6b6 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:17.885101 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:17.890890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265cb5a-65db-45de-be59-47533c78a6b6 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:17.896129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477771c-b2f9-46f1-a784-f266a45b3c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:17.896708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477771c-b2f9-46f1-a784-f266a45b3c5e admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:17.896984 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477771c-b2f9-46f1-a784-f266a45b3c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:17.897383 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477771c-b2f9-46f1-a784-f266a45b3c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:17.897989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477771c-b2f9-46f1-a784-f266a45b3c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:17.900130 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477771c-b2f9-46f1-a784-f266a45b3c5e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:17.910748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1162/2328] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:40:17.914811 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0daaf7-7659-4853-b437-5393cf8cac5a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:17.925562 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0daaf7-7659-4853-b437-5393cf8cac5a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:17.930692 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52594d0-2c85-4cc4-881b-5322b2b60b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:17.931231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52594d0-2c85-4cc4-881b-5322b2b60b87 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:17.931510 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52594d0-2c85-4cc4-881b-5322b2b60b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:17.931783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52594d0-2c85-4cc4-881b-5322b2b60b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:17.932431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52594d0-2c85-4cc4-881b-5322b2b60b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:17.934066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52594d0-2c85-4cc4-881b-5322b2b60b87 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:17.945461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1167/2329] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:17.953618 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477771c-b2f9-46f1-a784-f266a45b3c5e admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:17.962284 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477771c-b2f9-46f1-a784-f266a45b3c5e admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:17.968119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0b25a-d156-4474-a539-5bae5db1d19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:17.968856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0b25a-d156-4474-a539-5bae5db1d19e admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:17.969233 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0b25a-d156-4474-a539-5bae5db1d19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:17.969536 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0b25a-d156-4474-a539-5bae5db1d19e admin admin] PATH_INFO: `/v3/projects/6be6096fafbf457bb1d852f0a7a41155/users/a76803e229934fa795458335fc30d849/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:17.986171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0b25a-d156-4474-a539-5bae5db1d19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be6096fafbf457bb1d852f0a7a41155, user_id=a76803e229934fa795458335fc30d849, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:17.988769 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0b25a-d156-4474-a539-5bae5db1d19e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:18.033976 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1163/2330] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:17 2022] PUT /identity/v3/projects/6be6096fafbf457bb1d852f0a7a41155/users/a76803e229934fa795458335fc30d849/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:18.044737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273664f-eab1-4655-a055-1d3e839b1fc2 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:18.045090 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273664f-eab1-4655-a055-1d3e839b1fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:18.045370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273664f-eab1-4655-a055-1d3e839b1fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:18.073511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-0273664f-eab1-4655-a055-1d3e839b1fc2 None None] MFA Rules not processed for user `a76803e229934fa795458335fc30d849`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:18.074737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0273664f-eab1-4655-a055-1d3e839b1fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:18.146733 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1168/2331] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:18.157881 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0b25a-d156-4474-a539-5bae5db1d19e admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:18.159658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2ef0b25a-d156-4474-a539-5bae5db1d19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:18.218809 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0b25a-d156-4474-a539-5bae5db1d19e admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:18.222438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5463d-573d-42d5-a186-0a3fb7d68e8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:18.223069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5463d-573d-42d5-a186-0a3fb7d68e8e service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:18.223351 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5463d-573d-42d5-a186-0a3fb7d68e8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:18.224199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5463d-573d-42d5-a186-0a3fb7d68e8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:18.239315 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5463d-573d-42d5-a186-0a3fb7d68e8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:18.240991 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5463d-573d-42d5-a186-0a3fb7d68e8e service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:18.251495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1164/2332] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:21.736387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781f2e3-9ae7-43ba-acb3-55ba4cf6ac00 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:21.736851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781f2e3-9ae7-43ba-acb3-55ba4cf6ac00 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:21.737047 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781f2e3-9ae7-43ba-acb3-55ba4cf6ac00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:21.758583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-3781f2e3-9ae7-43ba-acb3-55ba4cf6ac00 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:21.760062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-3781f2e3-9ae7-43ba-acb3-55ba4cf6ac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:21.829052 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1169/2333] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:40:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:21.856390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5463d-573d-42d5-a186-0a3fb7d68e8e service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:21.857882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-22c5463d-573d-42d5-a186-0a3fb7d68e8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:21.930715 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5463d-573d-42d5-a186-0a3fb7d68e8e service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:21.935906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d37e1-bf3c-4649-98bd-63fa898f24aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:21.936509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d37e1-bf3c-4649-98bd-63fa898f24aa service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:21.936781 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d37e1-bf3c-4649-98bd-63fa898f24aa service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:21.937050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d37e1-bf3c-4649-98bd-63fa898f24aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:21.945406 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d37e1-bf3c-4649-98bd-63fa898f24aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:21.947055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d37e1-bf3c-4649-98bd-63fa898f24aa service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:21.955080 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1165/2334] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:40:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:40:29.862173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781f2e3-9ae7-43ba-acb3-55ba4cf6ac00 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:29.872112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781f2e3-9ae7-43ba-acb3-55ba4cf6ac00 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:29.875834 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb791c-b55f-4baa-8d5f-e3ddc3af36d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:29.876426 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb791c-b55f-4baa-8d5f-e3ddc3af36d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:29.876703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb791c-b55f-4baa-8d5f-e3ddc3af36d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:29.876978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb791c-b55f-4baa-8d5f-e3ddc3af36d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:29.878719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-70cb791c-b55f-4baa-8d5f-e3ddc3af36d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:29.938550 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:29.943441 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb791c-b55f-4baa-8d5f-e3ddc3af36d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:29.945362 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb791c-b55f-4baa-8d5f-e3ddc3af36d7 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:29.955353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1170/2335] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:31.952146 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=72,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:35.898743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d37e1-bf3c-4649-98bd-63fa898f24aa service swift] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:35.958508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d37e1-bf3c-4649-98bd-63fa898f24aa service swift] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:35.962977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:35.964198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:35.964393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:35.964657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] PATH_INFO: `/v3/users/a76803e229934fa795458335fc30d849` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:35.965752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76803e229934fa795458335fc30d849)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:35.967694 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:36.137339 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb791c-b55f-4baa-8d5f-e3ddc3af36d7 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:36.138727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-70cb791c-b55f-4baa-8d5f-e3ddc3af36d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:36.191275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76803e229934fa795458335fc30d849'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:36.192012 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76803e229934fa795458335fc30d849'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:36.228828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb791c-b55f-4baa-8d5f-e3ddc3af36d7 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:36.236016 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa37f9a-65d9-44b1-9a3a-597cee5d5272 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:36.236614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa37f9a-65d9-44b1-9a3a-597cee5d5272 service placement] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:36.236894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa37f9a-65d9-44b1-9a3a-597cee5d5272 service placement] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:36.237155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa37f9a-65d9-44b1-9a3a-597cee5d5272 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:36.238929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8aa37f9a-65d9-44b1-9a3a-597cee5d5272 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:36.251622 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa37f9a-65d9-44b1-9a3a-597cee5d5272 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:36.253182 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa37f9a-65d9-44b1-9a3a-597cee5d5272 service placement] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:36.263917 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1171/2336] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:36.297286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76803e229934fa795458335fc30d849'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:36.298371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76803e229934fa795458335fc30d849'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:36.311844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76803e229934fa795458335fc30d849'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:36.331640 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1166/2337] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:35 2022] DELETE /identity/v3/users/a76803e229934fa795458335fc30d849 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:36.336153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4f867-3bd6-4809-8c16-221885e5dc58 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:36.341632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4f867-3bd6-4809-8c16-221885e5dc58 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:36.341632 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4f867-3bd6-4809-8c16-221885e5dc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:36.355963 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-8dd4f867-3bd6-4809-8c16-221885e5dc58 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:36.357174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8dd4f867-3bd6-4809-8c16-221885e5dc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:36.418120 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1172/2338] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:36.430336 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:36.431897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:36.489604 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec11d55-1514-4269-be87-8d19a32d80da admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:36.493242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af4afc-7d31-46f3-bb56-ca063868a04a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:36.493843 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af4afc-7d31-46f3-bb56-ca063868a04a service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:36.494162 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af4afc-7d31-46f3-bb56-ca063868a04a service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:36.494437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af4afc-7d31-46f3-bb56-ca063868a04a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:36.504152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af4afc-7d31-46f3-bb56-ca063868a04a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:36.505576 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af4afc-7d31-46f3-bb56-ca063868a04a service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:36.515419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1167/2339] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:36.938833 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4f867-3bd6-4809-8c16-221885e5dc58 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:36.940376 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8dd4f867-3bd6-4809-8c16-221885e5dc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:36.954148 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4f867-3bd6-4809-8c16-221885e5dc58 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:36.971630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:36.971630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:36.971630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:36.971630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] PATH_INFO: `/v3/projects/6be6096fafbf457bb1d852f0a7a41155` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:36.972545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be6096fafbf457bb1d852f0a7a41155)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:36.974203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:37.103410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be6096fafbf457bb1d852f0a7a41155', 'request_id': 'req-b7b07c8d-5658-41ed-82d0-7b8befe3468f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:37.113282 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be6096fafbf457bb1d852f0a7a41155', 'request_id': 'req-b7b07c8d-5658-41ed-82d0-7b8befe3468f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:37.117852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:37.189201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be6096fafbf457bb1d852f0a7a41155', 'request_id': 'req-b7b07c8d-5658-41ed-82d0-7b8befe3468f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:37.203816 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] The token cache is being invalidate because project 6be6096fafbf457bb1d852f0a7a41155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:37.204108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:37.205030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7b07c8d-5658-41ed-82d0-7b8befe3468f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:37.206565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1173/2340] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:36 2022] DELETE /identity/v3/projects/6be6096fafbf457bb1d852f0a7a41155 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:37.211453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b17c77-590d-40a6-9a53-c290486adec4 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:37.212004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b17c77-590d-40a6-9a53-c290486adec4 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:37.212103 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b17c77-590d-40a6-9a53-c290486adec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:37.236844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-15b17c77-590d-40a6-9a53-c290486adec4 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:37.238203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-15b17c77-590d-40a6-9a53-c290486adec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:37.255763 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:37.303267 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1168/2341] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:37.370501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f473eb-731c-4304-a077-9f33e775f211 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:37.371047 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f473eb-731c-4304-a077-9f33e775f211 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:37.371047 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f473eb-731c-4304-a077-9f33e775f211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:37.393297 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-86f473eb-731c-4304-a077-9f33e775f211 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:37.394519 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-86f473eb-731c-4304-a077-9f33e775f211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:37.403761 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1174/2342] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:37.408138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b17c77-590d-40a6-9a53-c290486adec4 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:37.426615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b17c77-590d-40a6-9a53-c290486adec4 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:37.431650 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038fea3-cb5c-4f3c-a563-90aa6765b357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:37.432382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038fea3-cb5c-4f3c-a563-90aa6765b357 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:37.432596 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038fea3-cb5c-4f3c-a563-90aa6765b357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:37.432899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038fea3-cb5c-4f3c-a563-90aa6765b357 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:37.433671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038fea3-cb5c-4f3c-a563-90aa6765b357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:40:37.433977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038fea3-cb5c-4f3c-a563-90aa6765b357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:37.435760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038fea3-cb5c-4f3c-a563-90aa6765b357 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:37.449016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1169/2343] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:40:37.453566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f473eb-731c-4304-a077-9f33e775f211 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:37.464507 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f473eb-731c-4304-a077-9f33e775f211 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:37.469703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70de13-da84-4886-9f46-9e26339cbbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:37.470063 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70de13-da84-4886-9f46-9e26339cbbdb admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:37.470344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70de13-da84-4886-9f46-9e26339cbbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:37.470611 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70de13-da84-4886-9f46-9e26339cbbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:37.471241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70de13-da84-4886-9f46-9e26339cbbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:37.473120 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70de13-da84-4886-9f46-9e26339cbbdb admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:37.578740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1175/2344] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:37 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:37.582748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038fea3-cb5c-4f3c-a563-90aa6765b357 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:37.592452 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038fea3-cb5c-4f3c-a563-90aa6765b357 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:37.648622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f0a0b-bf47-45e4-b04d-f9e18ada858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:37.649230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f0a0b-bf47-45e4-b04d-f9e18ada858c admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:37.649562 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f0a0b-bf47-45e4-b04d-f9e18ada858c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:37.649832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f0a0b-bf47-45e4-b04d-f9e18ada858c admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:37.650551 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f0a0b-bf47-45e4-b04d-f9e18ada858c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:37.652791 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f0a0b-bf47-45e4-b04d-f9e18ada858c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:37.777663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1170/2345] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:37 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:37.781755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70de13-da84-4886-9f46-9e26339cbbdb admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:37.793247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70de13-da84-4886-9f46-9e26339cbbdb admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:37.797807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baff8ed-65d3-4fc5-8bfe-68212a34cef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:37.798549 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baff8ed-65d3-4fc5-8bfe-68212a34cef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:37.798826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baff8ed-65d3-4fc5-8bfe-68212a34cef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:37.799093 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baff8ed-65d3-4fc5-8bfe-68212a34cef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:37.799728 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baff8ed-65d3-4fc5-8bfe-68212a34cef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:37.801296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baff8ed-65d3-4fc5-8bfe-68212a34cef3 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:37.811778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1176/2346] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:37.816020 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f0a0b-bf47-45e4-b04d-f9e18ada858c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:37.827451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f0a0b-bf47-45e4-b04d-f9e18ada858c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:37.832072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac69732-5c6c-496d-943f-1c9b48965276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:37.832775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac69732-5c6c-496d-943f-1c9b48965276 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:37.833051 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac69732-5c6c-496d-943f-1c9b48965276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:37.833327 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac69732-5c6c-496d-943f-1c9b48965276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:37.834005 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac69732-5c6c-496d-943f-1c9b48965276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:37.835694 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac69732-5c6c-496d-943f-1c9b48965276 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:37.850866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1171/2347] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:37.852321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baff8ed-65d3-4fc5-8bfe-68212a34cef3 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:37.862254 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baff8ed-65d3-4fc5-8bfe-68212a34cef3 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:37.866936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04439eb-bcc0-4a33-a1be-1558df304289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:37.867533 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04439eb-bcc0-4a33-a1be-1558df304289 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:37.867806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04439eb-bcc0-4a33-a1be-1558df304289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:37.868075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04439eb-bcc0-4a33-a1be-1558df304289 admin admin] PATH_INFO: `/v3/projects/d0317a0126104e40908451f726e7dcdf/users/920f1d5cf1a643b7b7b7f4c3465c61d5/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:37.883808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04439eb-bcc0-4a33-a1be-1558df304289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0317a0126104e40908451f726e7dcdf, user_id=920f1d5cf1a643b7b7b7f4c3465c61d5, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:37.887247 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04439eb-bcc0-4a33-a1be-1558df304289 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:37.948402 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1177/2348] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:37 2022] PUT /identity/v3/projects/d0317a0126104e40908451f726e7dcdf/users/920f1d5cf1a643b7b7b7f4c3465c61d5/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:37.958504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48535403-ef84-4470-8cd6-8c74bf7a10fb None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:37.958787 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48535403-ef84-4470-8cd6-8c74bf7a10fb None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:37.959062 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48535403-ef84-4470-8cd6-8c74bf7a10fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:37.986564 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-48535403-ef84-4470-8cd6-8c74bf7a10fb None None] MFA Rules not processed for user `920f1d5cf1a643b7b7b7f4c3465c61d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:37.987760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-48535403-ef84-4470-8cd6-8c74bf7a10fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:38.061479 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1172/2349] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:38.062822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04439eb-bcc0-4a33-a1be-1558df304289 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:38.131053 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04439eb-bcc0-4a33-a1be-1558df304289 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:38.135910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dcbbd-3f0b-4630-b76f-38ff66d6cade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:38.140911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dcbbd-3f0b-4630-b76f-38ff66d6cade admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:38.140911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dcbbd-3f0b-4630-b76f-38ff66d6cade admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:38.140911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621dcbbd-3f0b-4630-b76f-38ff66d6cade admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:38.140911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dcbbd-3f0b-4630-b76f-38ff66d6cade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:38.140911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621dcbbd-3f0b-4630-b76f-38ff66d6cade admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:38.264965 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1178/2350] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:38 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:38.269180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48535403-ef84-4470-8cd6-8c74bf7a10fb None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:38.280240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48535403-ef84-4470-8cd6-8c74bf7a10fb None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:38.293462 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:38.339313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f05a2-56af-4037-95df-6678d0fce1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:38.341554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f05a2-56af-4037-95df-6678d0fce1ef admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:38.341554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f05a2-56af-4037-95df-6678d0fce1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:38.341554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f05a2-56af-4037-95df-6678d0fce1ef admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:38.341554 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f05a2-56af-4037-95df-6678d0fce1ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:38.343870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f05a2-56af-4037-95df-6678d0fce1ef admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:38.460934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1173/2351] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:38 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:38.462120 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dcbbd-3f0b-4630-b76f-38ff66d6cade admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:38.472926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621dcbbd-3f0b-4630-b76f-38ff66d6cade admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:38.477616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55fd16-1bef-443f-b5d1-829f1efcd588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:38.478311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55fd16-1bef-443f-b5d1-829f1efcd588 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:38.478588 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55fd16-1bef-443f-b5d1-829f1efcd588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:38.478857 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f55fd16-1bef-443f-b5d1-829f1efcd588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:38.481039 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55fd16-1bef-443f-b5d1-829f1efcd588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:38.482626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f55fd16-1bef-443f-b5d1-829f1efcd588 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:38.500012 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1179/2352] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:38.503304 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f05a2-56af-4037-95df-6678d0fce1ef admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:38.513384 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f05a2-56af-4037-95df-6678d0fce1ef admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:38.518049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae31a7-c907-4b28-b229-3dabffb87e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:38.518656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae31a7-c907-4b28-b229-3dabffb87e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:38.518931 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae31a7-c907-4b28-b229-3dabffb87e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:38.519205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae31a7-c907-4b28-b229-3dabffb87e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:38.519845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae31a7-c907-4b28-b229-3dabffb87e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:38.521433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae31a7-c907-4b28-b229-3dabffb87e0e admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:38.531720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1174/2353] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:38.535759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55fd16-1bef-443f-b5d1-829f1efcd588 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:38.546170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f55fd16-1bef-443f-b5d1-829f1efcd588 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:38.550740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9fdda-1070-4968-ae4b-efdd78ac7ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:38.551314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9fdda-1070-4968-ae4b-efdd78ac7ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:38.551594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9fdda-1070-4968-ae4b-efdd78ac7ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:38.551869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9fdda-1070-4968-ae4b-efdd78ac7ca0 admin admin] PATH_INFO: `/v3/projects/639dd1060d5043e982094872269651f8/users/cb93bb75049b40cb8081ed7fca3309d9/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:38.566116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9fdda-1070-4968-ae4b-efdd78ac7ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=639dd1060d5043e982094872269651f8, user_id=cb93bb75049b40cb8081ed7fca3309d9, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:38.568788 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9fdda-1070-4968-ae4b-efdd78ac7ca0 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:38.633802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1180/2354] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:38 2022] PUT /identity/v3/projects/639dd1060d5043e982094872269651f8/users/cb93bb75049b40cb8081ed7fca3309d9/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:38.644118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861b7ed-bfe6-4f76-a9e6-49796868b0df None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:38.644407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861b7ed-bfe6-4f76-a9e6-49796868b0df None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:38.644685 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861b7ed-bfe6-4f76-a9e6-49796868b0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:38.673366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-7861b7ed-bfe6-4f76-a9e6-49796868b0df None None] MFA Rules not processed for user `cb93bb75049b40cb8081ed7fca3309d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:38.674619 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7861b7ed-bfe6-4f76-a9e6-49796868b0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:38.739223 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1175/2355] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:38.743652 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9fdda-1070-4968-ae4b-efdd78ac7ca0 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:38.778900 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:38.805350 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9fdda-1070-4968-ae4b-efdd78ac7ca0 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:38.810499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f966d-95b2-4a27-9205-a1e82666f8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:38.811135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f966d-95b2-4a27-9205-a1e82666f8f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:38.811416 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f966d-95b2-4a27-9205-a1e82666f8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:38.811689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f966d-95b2-4a27-9205-a1e82666f8f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:38.812329 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f966d-95b2-4a27-9205-a1e82666f8f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:38.814248 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f966d-95b2-4a27-9205-a1e82666f8f3 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:38.875055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1181/2356] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:38 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:40:38.879386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861b7ed-bfe6-4f76-a9e6-49796868b0df None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:38.890579 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861b7ed-bfe6-4f76-a9e6-49796868b0df None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:38.944887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcab05-7074-427a-ae1b-e96cb640d34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:38.945497 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcab05-7074-427a-ae1b-e96cb640d34a admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:38.945779 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcab05-7074-427a-ae1b-e96cb640d34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:38.946109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcab05-7074-427a-ae1b-e96cb640d34a admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:38.946770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bcab05-7074-427a-ae1b-e96cb640d34a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:38.949212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bcab05-7074-427a-ae1b-e96cb640d34a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:39.028176 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1176/2357] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:38 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:40:39.032232 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f966d-95b2-4a27-9205-a1e82666f8f3 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:39.042458 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f966d-95b2-4a27-9205-a1e82666f8f3 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:39.047096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe10fa-7a09-4d5c-add8-6021c8186c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:39.047650 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe10fa-7a09-4d5c-add8-6021c8186c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:39.047928 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe10fa-7a09-4d5c-add8-6021c8186c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:39.048198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe10fa-7a09-4d5c-add8-6021c8186c7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:39.048836 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe10fa-7a09-4d5c-add8-6021c8186c7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:39.050461 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe10fa-7a09-4d5c-add8-6021c8186c7e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:39.060668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1182/2358] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:39.064403 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcab05-7074-427a-ae1b-e96cb640d34a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:39.074684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcab05-7074-427a-ae1b-e96cb640d34a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:39.079229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf98c8-c10d-44ed-9e49-1d83138ae62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:39.079866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf98c8-c10d-44ed-9e49-1d83138ae62c admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:39.080145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf98c8-c10d-44ed-9e49-1d83138ae62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:39.080418 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf98c8-c10d-44ed-9e49-1d83138ae62c admin admin] PATH_INFO: `/v3/projects/31ce763de2554b70b62a624e8182ce97/users/39056fc103c445ad823b95eac35b6356/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:39.092084 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:39.095016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf98c8-c10d-44ed-9e49-1d83138ae62c admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ce763de2554b70b62a624e8182ce97, user_id=39056fc103c445ad823b95eac35b6356, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:39.097701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf98c8-c10d-44ed-9e49-1d83138ae62c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:39.153893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1177/2359] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:39 2022] PUT /identity/v3/projects/31ce763de2554b70b62a624e8182ce97/users/39056fc103c445ad823b95eac35b6356/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:39.158129 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe10fa-7a09-4d5c-add8-6021c8186c7e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:39.168154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe10fa-7a09-4d5c-add8-6021c8186c7e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:39.222758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1645a6-b182-4566-b5b4-4dbf736b790b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:39.223356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1645a6-b182-4566-b5b4-4dbf736b790b admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:39.223634 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1645a6-b182-4566-b5b4-4dbf736b790b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:39.223906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1645a6-b182-4566-b5b4-4dbf736b790b admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:39.224537 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1645a6-b182-4566-b5b4-4dbf736b790b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:39.226161 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1645a6-b182-4566-b5b4-4dbf736b790b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:39.235565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1183/2360] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:39.239780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf98c8-c10d-44ed-9e49-1d83138ae62c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:39.251260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf98c8-c10d-44ed-9e49-1d83138ae62c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:39.255880 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13690167-16f1-4126-aa41-d2ee42b1a7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:39.256517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13690167-16f1-4126-aa41-d2ee42b1a7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:39.256805 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13690167-16f1-4126-aa41-d2ee42b1a7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:39.257055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13690167-16f1-4126-aa41-d2ee42b1a7ab admin admin] PATH_INFO: `/v3/projects/31ce763de2554b70b62a624e8182ce97/users/39056fc103c445ad823b95eac35b6356/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:39.259004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13690167-16f1-4126-aa41-d2ee42b1a7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ce763de2554b70b62a624e8182ce97, user_id=39056fc103c445ad823b95eac35b6356, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:39.262214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13690167-16f1-4126-aa41-d2ee42b1a7ab admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:39.272467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1178/2361] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:39 2022] PUT /identity/v3/projects/31ce763de2554b70b62a624e8182ce97/users/39056fc103c445ad823b95eac35b6356/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:39.284102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57162e3d-a777-4f79-9af5-046190e8bd97 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:39.284513 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57162e3d-a777-4f79-9af5-046190e8bd97 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:39.284726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57162e3d-a777-4f79-9af5-046190e8bd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:39.313387 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-57162e3d-a777-4f79-9af5-046190e8bd97 None None] MFA Rules not processed for user `39056fc103c445ad823b95eac35b6356`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:39.314618 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-57162e3d-a777-4f79-9af5-046190e8bd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:39.386755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1184/2362] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:39.398448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13690167-16f1-4126-aa41-d2ee42b1a7ab admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:39.400161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-13690167-16f1-4126-aa41-d2ee42b1a7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:39.459107 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13690167-16f1-4126-aa41-d2ee42b1a7ab admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:39.462889 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9ceca-d518-4de7-b7fa-115458b6ebed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:39.463542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9ceca-d518-4de7-b7fa-115458b6ebed service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:39.463833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9ceca-d518-4de7-b7fa-115458b6ebed service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:39.464090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9ceca-d518-4de7-b7fa-115458b6ebed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:39.520938 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9ceca-d518-4de7-b7fa-115458b6ebed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:39.522412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9ceca-d518-4de7-b7fa-115458b6ebed service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:39.531496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1179/2363] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:40.507783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1adf-42a5-4c9c-b891-5e5b31ac2b2a None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:40.508299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1adf-42a5-4c9c-b891-5e5b31ac2b2a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:40.508431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527f1adf-42a5-4c9c-b891-5e5b31ac2b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:40.528722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-527f1adf-42a5-4c9c-b891-5e5b31ac2b2a None None] MFA Rules not processed for user `0bd6670e1a0d46b088c13e82e31dc6da`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:40.529992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-527f1adf-42a5-4c9c-b891-5e5b31ac2b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:40.554644 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:40.586499 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1185/2364] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:40.700042 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9ceca-d518-4de7-b7fa-115458b6ebed service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:40.701446 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f5d9ceca-d518-4de7-b7fa-115458b6ebed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:40.768972 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9ceca-d518-4de7-b7fa-115458b6ebed service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:40.774311 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:40.774970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:40.775245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:40.775517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] PATH_INFO: `/v3/users/1c3808cb27af418ebe5137d2c439ac97` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:40.777202 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c3808cb27af418ebe5137d2c439ac97)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:40.779190 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:40.994071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:41.004916 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c3808cb27af418ebe5137d2c439ac97'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:41.006234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c3808cb27af418ebe5137d2c439ac97'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:41.062230 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c3808cb27af418ebe5137d2c439ac97'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:41.063018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c3808cb27af418ebe5137d2c439ac97'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:41.071747 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c3808cb27af418ebe5137d2c439ac97'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:41.100024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994688b2-fdc2-4a26-a3a4-792d668d17f6 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:41.103327 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994688b2-fdc2-4a26-a3a4-792d668d17f6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:41.103869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994688b2-fdc2-4a26-a3a4-792d668d17f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:41.110135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1180/2365] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:40 2022] DELETE /identity/v3/users/1c3808cb27af418ebe5137d2c439ac97 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:41.143862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-994688b2-fdc2-4a26-a3a4-792d668d17f6 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:41.143862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-994688b2-fdc2-4a26-a3a4-792d668d17f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:41.188523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1186/2366] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:41.198831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:41.200207 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:41.260928 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483b35a-2032-422f-8734-aab617f608eb admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:41.264702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9c2bb-a7ce-4fea-9770-0f88bbd637f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:41.265386 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9c2bb-a7ce-4fea-9770-0f88bbd637f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:41.265684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9c2bb-a7ce-4fea-9770-0f88bbd637f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:41.266008 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9c2bb-a7ce-4fea-9770-0f88bbd637f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:41.276675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9c2bb-a7ce-4fea-9770-0f88bbd637f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:41.283337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9c2bb-a7ce-4fea-9770-0f88bbd637f7 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:41.289847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1181/2367] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:40:41.562959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994688b2-fdc2-4a26-a3a4-792d668d17f6 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:41.564530 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-994688b2-fdc2-4a26-a3a4-792d668d17f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:41.647484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994688b2-fdc2-4a26-a3a4-792d668d17f6 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:41.651583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf32e8e-53e4-437f-9555-31d4a3086d9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:41.652195 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf32e8e-53e4-437f-9555-31d4a3086d9e service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:41.652471 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf32e8e-53e4-437f-9555-31d4a3086d9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:41.652737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf32e8e-53e4-437f-9555-31d4a3086d9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:41.654580 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5cf32e8e-53e4-437f-9555-31d4a3086d9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:41.714692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9c2bb-a7ce-4fea-9770-0f88bbd637f7 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:41.716281 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-70e9c2bb-a7ce-4fea-9770-0f88bbd637f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:41.724062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf32e8e-53e4-437f-9555-31d4a3086d9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:41.725253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf32e8e-53e4-437f-9555-31d4a3086d9e service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:41.736234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1187/2368] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:41.753875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9c2bb-a7ce-4fea-9770-0f88bbd637f7 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:41.766237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:41.768502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:41.768502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:41.768502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] PATH_INFO: `/v3/projects/80abbd8aca9646e7ad9fba1e79900613` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:41.769481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80abbd8aca9646e7ad9fba1e79900613)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:41.774702 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:41.929064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80abbd8aca9646e7ad9fba1e79900613', 'request_id': 'req-6e06b212-75a9-4157-80e0-cdb96f8d31d9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:41.936154 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80abbd8aca9646e7ad9fba1e79900613', 'request_id': 'req-6e06b212-75a9-4157-80e0-cdb96f8d31d9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:42.017093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80abbd8aca9646e7ad9fba1e79900613', 'request_id': 'req-6e06b212-75a9-4157-80e0-cdb96f8d31d9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:42.032832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] The token cache is being invalidate because project 80abbd8aca9646e7ad9fba1e79900613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:42.033193 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:42.034443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6e06b212-75a9-4157-80e0-cdb96f8d31d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:42.035838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1182/2369] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:41 2022] DELETE /identity/v3/projects/80abbd8aca9646e7ad9fba1e79900613 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:40:42.043843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6ef10-0b67-4efa-85d0-ccf87ec5ae42 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:42.043843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6ef10-0b67-4efa-85d0-ccf87ec5ae42 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:42.043843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6ef10-0b67-4efa-85d0-ccf87ec5ae42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:42.063730 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-0ac6ef10-0b67-4efa-85d0-ccf87ec5ae42 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:42.065064 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0ac6ef10-0b67-4efa-85d0-ccf87ec5ae42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:42.091946 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:42.150183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1188/2370] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:42.158605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d280f-42c7-4b8b-b600-5723f6e30e7b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:42.158937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d280f-42c7-4b8b-b600-5723f6e30e7b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:42.159276 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d280f-42c7-4b8b-b600-5723f6e30e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:42.183503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c07d280f-42c7-4b8b-b600-5723f6e30e7b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:42.183503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c07d280f-42c7-4b8b-b600-5723f6e30e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:42.189792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1183/2371] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:40:42.194114 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6ef10-0b67-4efa-85d0-ccf87ec5ae42 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:42.204312 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6ef10-0b67-4efa-85d0-ccf87ec5ae42 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:42.209213 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ba95b-de6e-4d23-95db-8e3b901228e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:42.209809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ba95b-de6e-4d23-95db-8e3b901228e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:42.210115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ba95b-de6e-4d23-95db-8e3b901228e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:42.210423 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ba95b-de6e-4d23-95db-8e3b901228e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:42.211075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ba95b-de6e-4d23-95db-8e3b901228e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:40:42.211401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ba95b-de6e-4d23-95db-8e3b901228e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:42.213061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ba95b-de6e-4d23-95db-8e3b901228e9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:42.232526 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1189/2372] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:40:42.236784 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d280f-42c7-4b8b-b600-5723f6e30e7b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:42.246953 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d280f-42c7-4b8b-b600-5723f6e30e7b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:42.252174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f432e-cd8c-456a-81ed-3b581db812c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:42.252797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f432e-cd8c-456a-81ed-3b581db812c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:42.253100 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f432e-cd8c-456a-81ed-3b581db812c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:42.253374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f432e-cd8c-456a-81ed-3b581db812c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:42.254024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f432e-cd8c-456a-81ed-3b581db812c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:42.256225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f432e-cd8c-456a-81ed-3b581db812c5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:42.347320 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1184/2373] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:42 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:42.356722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ba95b-de6e-4d23-95db-8e3b901228e9 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:42.366803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ba95b-de6e-4d23-95db-8e3b901228e9 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:42.427166 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76d2ef-ac5b-4b6a-a118-0d81431f0ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:42.427768 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76d2ef-ac5b-4b6a-a118-0d81431f0ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:42.428043 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76d2ef-ac5b-4b6a-a118-0d81431f0ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:42.428312 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76d2ef-ac5b-4b6a-a118-0d81431f0ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:42.428954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76d2ef-ac5b-4b6a-a118-0d81431f0ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:42.430920 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76d2ef-ac5b-4b6a-a118-0d81431f0ba6 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:42.523407 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1190/2374] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:42 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:42.524736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f432e-cd8c-456a-81ed-3b581db812c5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:42.535092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f432e-cd8c-456a-81ed-3b581db812c5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:42.540299 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d513e3-842a-4f3a-bca3-322c826acb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:42.549509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d513e3-842a-4f3a-bca3-322c826acb73 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:42.549509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d513e3-842a-4f3a-bca3-322c826acb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:42.549509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d513e3-842a-4f3a-bca3-322c826acb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:42.549509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d513e3-842a-4f3a-bca3-322c826acb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:42.549509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d513e3-842a-4f3a-bca3-322c826acb73 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:42.549509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:42.556173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1185/2375] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:42.560793 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76d2ef-ac5b-4b6a-a118-0d81431f0ba6 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:42.571328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76d2ef-ac5b-4b6a-a118-0d81431f0ba6 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:42.577188 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6e1a2-5ee8-4dbc-9f43-cd5001296534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:42.578086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6e1a2-5ee8-4dbc-9f43-cd5001296534 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:42.578591 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6e1a2-5ee8-4dbc-9f43-cd5001296534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:42.578845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6e1a2-5ee8-4dbc-9f43-cd5001296534 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:42.579498 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6e1a2-5ee8-4dbc-9f43-cd5001296534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:42.581102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6e1a2-5ee8-4dbc-9f43-cd5001296534 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:42.592208 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1191/2376] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:42.597004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d513e3-842a-4f3a-bca3-322c826acb73 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:42.609222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d513e3-842a-4f3a-bca3-322c826acb73 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:42.614232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89072bb-7ce9-4e51-8e73-9a7af4325067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:42.614960 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89072bb-7ce9-4e51-8e73-9a7af4325067 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:42.615321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89072bb-7ce9-4e51-8e73-9a7af4325067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:42.615663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89072bb-7ce9-4e51-8e73-9a7af4325067 admin admin] PATH_INFO: `/v3/projects/fe3a006650604404841394175e2c4385/users/2b526164131449f6a1f41175c606695d/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:42.632777 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89072bb-7ce9-4e51-8e73-9a7af4325067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3a006650604404841394175e2c4385, user_id=2b526164131449f6a1f41175c606695d, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:42.635428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89072bb-7ce9-4e51-8e73-9a7af4325067 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:42.738934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1186/2377] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:42 2022] PUT /identity/v3/projects/fe3a006650604404841394175e2c4385/users/2b526164131449f6a1f41175c606695d/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:42.748564 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911b2b8-33c7-4885-97e9-2b623c985df0 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:42.748851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911b2b8-33c7-4885-97e9-2b623c985df0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:42.749157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911b2b8-33c7-4885-97e9-2b623c985df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:42.779584 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-6911b2b8-33c7-4885-97e9-2b623c985df0 None None] MFA Rules not processed for user `2b526164131449f6a1f41175c606695d`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:42.781305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6911b2b8-33c7-4885-97e9-2b623c985df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:42.850147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1192/2378] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:42.861504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89072bb-7ce9-4e51-8e73-9a7af4325067 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:42.863728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a89072bb-7ce9-4e51-8e73-9a7af4325067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:42.920704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89072bb-7ce9-4e51-8e73-9a7af4325067 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:42.924874 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37205833-bf01-4649-adb8-90a848bb9a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:42.925491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205833-bf01-4649-adb8-90a848bb9a28 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:42.925904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205833-bf01-4649-adb8-90a848bb9a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:42.926217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205833-bf01-4649-adb8-90a848bb9a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:42.935297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37205833-bf01-4649-adb8-90a848bb9a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:42.936758 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37205833-bf01-4649-adb8-90a848bb9a28 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:42.946516 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1187/2379] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:43.270293 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911b2b8-33c7-4885-97e9-2b623c985df0 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:43.271683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-6911b2b8-33c7-4885-97e9-2b623c985df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:43.284978 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911b2b8-33c7-4885-97e9-2b623c985df0 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:43.288570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5c60b-0006-44ee-b02c-0f9ba8a40f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:43.289206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5c60b-0006-44ee-b02c-0f9ba8a40f32 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:43.289522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5c60b-0006-44ee-b02c-0f9ba8a40f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:43.289799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5c60b-0006-44ee-b02c-0f9ba8a40f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:43.291570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2ca5c60b-0006-44ee-b02c-0f9ba8a40f32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:43.314719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:43.369048 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5c60b-0006-44ee-b02c-0f9ba8a40f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:43.369460 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5c60b-0006-44ee-b02c-0f9ba8a40f32 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:43.381108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1193/2380] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:44.968883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37205833-bf01-4649-adb8-90a848bb9a28 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:44.971697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-37205833-bf01-4649-adb8-90a848bb9a28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:45.050831 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37205833-bf01-4649-adb8-90a848bb9a28 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:45.054599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdc332-aa51-4b35-9bfb-e87868354378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:45.055208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdc332-aa51-4b35-9bfb-e87868354378 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:45.055706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdc332-aa51-4b35-9bfb-e87868354378 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:45.056157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdc332-aa51-4b35-9bfb-e87868354378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:45.063605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9dcdc332-aa51-4b35-9bfb-e87868354378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:45.142073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdc332-aa51-4b35-9bfb-e87868354378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:45.143180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdc332-aa51-4b35-9bfb-e87868354378 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:45.153966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1188/2381] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:46.596647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80169f-fc71-47d2-91d3-42024ce8b603 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:46.597170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80169f-fc71-47d2-91d3-42024ce8b603 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:46.597283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b80169f-fc71-47d2-91d3-42024ce8b603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:46.654393 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-2b80169f-fc71-47d2-91d3-42024ce8b603 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:46.655706 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2b80169f-fc71-47d2-91d3-42024ce8b603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:46.761666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1194/2382] 10.222.0.20 () {58 vars in 967 bytes} [Wed Oct 19 02:40:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:46.770323 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdc332-aa51-4b35-9bfb-e87868354378 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:46.772764 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-9dcdc332-aa51-4b35-9bfb-e87868354378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:46.812758 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:46.855431 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdc332-aa51-4b35-9bfb-e87868354378 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:46.861957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34efbe-4ca0-4e11-ba59-28d57dff549e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:46.862715 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34efbe-4ca0-4e11-ba59-28d57dff549e service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:46.863692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34efbe-4ca0-4e11-ba59-28d57dff549e service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:46.863994 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34efbe-4ca0-4e11-ba59-28d57dff549e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:46.879257 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34efbe-4ca0-4e11-ba59-28d57dff549e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:46.881962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34efbe-4ca0-4e11-ba59-28d57dff549e service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:46.899066 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1189/2383] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:40:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:40:47.643415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80169f-fc71-47d2-91d3-42024ce8b603 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:47.718390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b80169f-fc71-47d2-91d3-42024ce8b603 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:47.723190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:47.723920 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:47.726842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:47.726842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] PATH_INFO: `/v3/users/2b526164131449f6a1f41175c606695d` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:47.726842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b526164131449f6a1f41175c606695d)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:47.727918 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:48.129973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b526164131449f6a1f41175c606695d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:48.130879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b526164131449f6a1f41175c606695d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:48.138246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b526164131449f6a1f41175c606695d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:48.194065 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b526164131449f6a1f41175c606695d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:48.194727 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b526164131449f6a1f41175c606695d'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:48.197087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:48.215438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1195/2384] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:47 2022] DELETE /identity/v3/users/2b526164131449f6a1f41175c606695d => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:48.220045 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6db87-8364-40f3-8223-d89315af9f1a None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:48.220572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6db87-8364-40f3-8223-d89315af9f1a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:48.220572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6db87-8364-40f3-8223-d89315af9f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:48.243203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-83f6db87-8364-40f3-8223-d89315af9f1a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:48.246799 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-83f6db87-8364-40f3-8223-d89315af9f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:48.328212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1190/2385] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:48.339381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:48.341273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:48.441139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3a552-21d2-4fef-9b7a-671ebfb484e1 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:48.444778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e8484-478a-4fc2-a0f3-44920367602f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:48.445391 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e8484-478a-4fc2-a0f3-44920367602f service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:48.445926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e8484-478a-4fc2-a0f3-44920367602f service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:48.446280 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e8484-478a-4fc2-a0f3-44920367602f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:48.457009 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e8484-478a-4fc2-a0f3-44920367602f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:48.458490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e8484-478a-4fc2-a0f3-44920367602f service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:48.479336 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1196/2386] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:48.547989 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6db87-8364-40f3-8223-d89315af9f1a None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:48.549592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-83f6db87-8364-40f3-8223-d89315af9f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:48.567531 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6db87-8364-40f3-8223-d89315af9f1a None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:48.590094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:48.590937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:48.591138 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:48.591417 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] PATH_INFO: `/v3/users/920f1d5cf1a643b7b7b7f4c3465c61d5` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:48.594756 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=920f1d5cf1a643b7b7b7f4c3465c61d5)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:48.598847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:48.815651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920f1d5cf1a643b7b7b7f4c3465c61d5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:48.816467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920f1d5cf1a643b7b7b7f4c3465c61d5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:48.925410 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920f1d5cf1a643b7b7b7f4c3465c61d5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:48.926471 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920f1d5cf1a643b7b7b7f4c3465c61d5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:48.940545 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920f1d5cf1a643b7b7b7f4c3465c61d5'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:48.968407 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1191/2387] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:48 2022] DELETE /identity/v3/users/920f1d5cf1a643b7b7b7f4c3465c61d5 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:48.972898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc907e6-b8bf-481c-a279-1365f1e253ec None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:48.973565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc907e6-b8bf-481c-a279-1365f1e253ec None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:48.973565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc907e6-b8bf-481c-a279-1365f1e253ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:48.997611 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-dfc907e6-b8bf-481c-a279-1365f1e253ec None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:48.998968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-dfc907e6-b8bf-481c-a279-1365f1e253ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:49.014123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:49.015532 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:49.084945 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:49.098181 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1197/2388] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:49.108346 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc907e6-b8bf-481c-a279-1365f1e253ec None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:49.110352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-dfc907e6-b8bf-481c-a279-1365f1e253ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:49.135655 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ff29a-d38e-45b4-adb0-c0aa2a9487c5 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:49.141661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:49.142310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:49.142663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:49.143007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] PATH_INFO: `/v3/projects/fe3a006650604404841394175e2c4385` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:49.144194 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3a006650604404841394175e2c4385)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:49.146668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:49.180941 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc907e6-b8bf-481c-a279-1365f1e253ec None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:49.185678 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb99dae-a12b-457c-9fcc-cf69bea8b24f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:49.186381 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb99dae-a12b-457c-9fcc-cf69bea8b24f service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:49.186656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb99dae-a12b-457c-9fcc-cf69bea8b24f service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:49.186923 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb99dae-a12b-457c-9fcc-cf69bea8b24f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:49.197273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb99dae-a12b-457c-9fcc-cf69bea8b24f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:49.198775 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb99dae-a12b-457c-9fcc-cf69bea8b24f service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:49.208236 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1198/2389] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:40:49.246461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3a006650604404841394175e2c4385', 'request_id': 'req-39b5861c-0568-4a3f-abb0-8186a46e4496'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:49.258009 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3a006650604404841394175e2c4385', 'request_id': 'req-39b5861c-0568-4a3f-abb0-8186a46e4496'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:49.342552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3a006650604404841394175e2c4385', 'request_id': 'req-39b5861c-0568-4a3f-abb0-8186a46e4496'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:49.354024 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] The token cache is being invalidate because project fe3a006650604404841394175e2c4385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:49.354339 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:49.354976 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-39b5861c-0568-4a3f-abb0-8186a46e4496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:49.356939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1192/2390] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:49 2022] DELETE /identity/v3/projects/fe3a006650604404841394175e2c4385 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:49.362469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6013c7-fdd1-43ee-98a1-3a8a9088ca61 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:49.362753 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6013c7-fdd1-43ee-98a1-3a8a9088ca61 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:49.363024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6013c7-fdd1-43ee-98a1-3a8a9088ca61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:49.385778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-5d6013c7-fdd1-43ee-98a1-3a8a9088ca61 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:49.387196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5d6013c7-fdd1-43ee-98a1-3a8a9088ca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:49.422563 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:49.474160 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1199/2391] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:49.475945 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac21a3-cc23-4100-8e7a-980a3041ba0f None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:49.476237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac21a3-cc23-4100-8e7a-980a3041ba0f None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:49.476572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac21a3-cc23-4100-8e7a-980a3041ba0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:49.497155 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-5fac21a3-cc23-4100-8e7a-980a3041ba0f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:49.498396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-5fac21a3-cc23-4100-8e7a-980a3041ba0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:49.507614 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1193/2392] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:40:49.512147 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6013c7-fdd1-43ee-98a1-3a8a9088ca61 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:49.521846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6013c7-fdd1-43ee-98a1-3a8a9088ca61 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:49.527517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9b74d-c039-49fb-b198-1fc81bd56181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:49.528328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9b74d-c039-49fb-b198-1fc81bd56181 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:49.533283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9b74d-c039-49fb-b198-1fc81bd56181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:49.533283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9b74d-c039-49fb-b198-1fc81bd56181 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:49.533283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9b74d-c039-49fb-b198-1fc81bd56181 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:40:49.533283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9b74d-c039-49fb-b198-1fc81bd56181 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:49.533283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9b74d-c039-49fb-b198-1fc81bd56181 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:49.543241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1200/2393] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:40:49.547849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac21a3-cc23-4100-8e7a-980a3041ba0f None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:49.560073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac21a3-cc23-4100-8e7a-980a3041ba0f None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:49.566140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71513b-b593-4ee8-aa79-0a02ba4a3a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:49.567726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71513b-b593-4ee8-aa79-0a02ba4a3a60 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:49.567726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71513b-b593-4ee8-aa79-0a02ba4a3a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:49.567726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71513b-b593-4ee8-aa79-0a02ba4a3a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:49.568077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71513b-b593-4ee8-aa79-0a02ba4a3a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:49.569915 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71513b-b593-4ee8-aa79-0a02ba4a3a60 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:49.646770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9b74d-c039-49fb-b198-1fc81bd56181 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:49.648568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-fbd9b74d-c039-49fb-b198-1fc81bd56181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:49.662561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9b74d-c039-49fb-b198-1fc81bd56181 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:49.668312 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:49.669448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:49.670024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:49.670325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] PATH_INFO: `/v3/projects/d0317a0126104e40908451f726e7dcdf` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:49.671710 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0317a0126104e40908451f726e7dcdf)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:49.673758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:49.710307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1194/2394] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:49 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:49.719133 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71513b-b593-4ee8-aa79-0a02ba4a3a60 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:49.824140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71513b-b593-4ee8-aa79-0a02ba4a3a60 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:49.825185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0317a0126104e40908451f726e7dcdf', 'request_id': 'req-010018b2-126d-444c-af7f-11992509cd4a'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:49.834373 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0317a0126104e40908451f726e7dcdf', 'request_id': 'req-010018b2-126d-444c-af7f-11992509cd4a'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:49.866683 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:49.883050 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cfc1c-e25d-4ae4-9bff-71f5314ae6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:49.883683 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cfc1c-e25d-4ae4-9bff-71f5314ae6b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:49.883966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cfc1c-e25d-4ae4-9bff-71f5314ae6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:49.884242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6cfc1c-e25d-4ae4-9bff-71f5314ae6b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:49.884875 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6cfc1c-e25d-4ae4-9bff-71f5314ae6b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:49.886887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6cfc1c-e25d-4ae4-9bff-71f5314ae6b1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:49.902935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0317a0126104e40908451f726e7dcdf', 'request_id': 'req-010018b2-126d-444c-af7f-11992509cd4a'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:49.913800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] The token cache is being invalidate because project d0317a0126104e40908451f726e7dcdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:49.914115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:49.914745 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:49.916860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1201/2395] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:49 2022] DELETE /identity/v3/projects/d0317a0126104e40908451f726e7dcdf => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:49.920601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:49.921897 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:49.935005 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010018b2-126d-444c-af7f-11992509cd4a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:49.940414 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:49.940860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:49.941190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:49.948154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] PATH_INFO: `/v3/users/cb93bb75049b40cb8081ed7fca3309d9` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:49.948154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb93bb75049b40cb8081ed7fca3309d9)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:49.948154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:49.969659 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1195/2396] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:49 2022] POST /identity/v3/users => generated 291 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:49.973605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cfc1c-e25d-4ae4-9bff-71f5314ae6b1 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:49.975456 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2e6cfc1c-e25d-4ae4-9bff-71f5314ae6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:49.987725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6cfc1c-e25d-4ae4-9bff-71f5314ae6b1 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:49.992669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a863d-9c6d-4b9e-ae94-3bcbb0a64cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:49.993111 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a863d-9c6d-4b9e-ae94-3bcbb0a64cdd admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:49.993376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a863d-9c6d-4b9e-ae94-3bcbb0a64cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:49.993648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48a863d-9c6d-4b9e-ae94-3bcbb0a64cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:49.994374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a863d-9c6d-4b9e-ae94-3bcbb0a64cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:49.996018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48a863d-9c6d-4b9e-ae94-3bcbb0a64cdd admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:50.006863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1196/2397] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:50.011453 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a863d-9c6d-4b9e-ae94-3bcbb0a64cdd admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:50.021646 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48a863d-9c6d-4b9e-ae94-3bcbb0a64cdd admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:50.027252 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c54e24-d153-4239-b1ef-e9d6d674ac73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:50.027939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c54e24-d153-4239-b1ef-e9d6d674ac73 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:50.028222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c54e24-d153-4239-b1ef-e9d6d674ac73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:50.028502 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c54e24-d153-4239-b1ef-e9d6d674ac73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:50.029152 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c54e24-d153-4239-b1ef-e9d6d674ac73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:50.031142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c54e24-d153-4239-b1ef-e9d6d674ac73 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:50.042899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1197/2398] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:50.047195 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c54e24-d153-4239-b1ef-e9d6d674ac73 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:50.057785 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c54e24-d153-4239-b1ef-e9d6d674ac73 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:50.062595 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e4cd3-937d-4e4c-a6cb-d95860f113ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:50.063260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e4cd3-937d-4e4c-a6cb-d95860f113ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:50.063538 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e4cd3-937d-4e4c-a6cb-d95860f113ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:50.063811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e4cd3-937d-4e4c-a6cb-d95860f113ad admin admin] PATH_INFO: `/v3/projects/9fb22a6e7d3244b2b1b3b5b1a3b9a963/users/4770ff701ece49d894cc910e40b32733/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:50.080789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e4cd3-937d-4e4c-a6cb-d95860f113ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb22a6e7d3244b2b1b3b5b1a3b9a963, user_id=4770ff701ece49d894cc910e40b32733, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:50.083523 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e4cd3-937d-4e4c-a6cb-d95860f113ad admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:50.118078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb93bb75049b40cb8081ed7fca3309d9'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:50.118696 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb93bb75049b40cb8081ed7fca3309d9'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:50.129832 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb93bb75049b40cb8081ed7fca3309d9'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:50.182974 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1198/2399] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:50 2022] PUT /identity/v3/projects/9fb22a6e7d3244b2b1b3b5b1a3b9a963/users/4770ff701ece49d894cc910e40b32733/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:40:50.188078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d50e0-e0f3-4215-b64b-f49f6bfe744f None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:50.188650 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d50e0-e0f3-4215-b64b-f49f6bfe744f None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:50.189118 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d50e0-e0f3-4215-b64b-f49f6bfe744f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:50.194129 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb93bb75049b40cb8081ed7fca3309d9'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:50.195130 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb93bb75049b40cb8081ed7fca3309d9'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:50.210893 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-fe7d50e0-e0f3-4215-b64b-f49f6bfe744f None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:50.212141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-fe7d50e0-e0f3-4215-b64b-f49f6bfe744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:50.217013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1202/2400] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:49 2022] DELETE /identity/v3/users/cb93bb75049b40cb8081ed7fca3309d9 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:40:50.290846 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1199/2401] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:40:50.303676 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:50.305116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:50.319715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:50.363106 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd7b21-c910-4ba6-9f9a-f4b456317ebf admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:50.366822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2edbe5-91c8-4794-b070-6e3562dd542e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:50.367425 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2edbe5-91c8-4794-b070-6e3562dd542e service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:50.367700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2edbe5-91c8-4794-b070-6e3562dd542e service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:50.367970 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2edbe5-91c8-4794-b070-6e3562dd542e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:50.431546 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2edbe5-91c8-4794-b070-6e3562dd542e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:50.433612 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2edbe5-91c8-4794-b070-6e3562dd542e service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:50.448266 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1203/2402] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:40:50.653499 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d50e0-e0f3-4215-b64b-f49f6bfe744f None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:50.655418 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-fe7d50e0-e0f3-4215-b64b-f49f6bfe744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:50.672993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d50e0-e0f3-4215-b64b-f49f6bfe744f None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:50.679313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:50.680929 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:50.681458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:50.682737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] PATH_INFO: `/v3/projects/639dd1060d5043e982094872269651f8` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:50.684296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=639dd1060d5043e982094872269651f8)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:50.686497 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:50.849745 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639dd1060d5043e982094872269651f8', 'request_id': 'req-8d20dec6-dc63-439d-a89f-ca9aebda54b7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:50.859193 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639dd1060d5043e982094872269651f8', 'request_id': 'req-8d20dec6-dc63-439d-a89f-ca9aebda54b7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:50.924987 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639dd1060d5043e982094872269651f8', 'request_id': 'req-8d20dec6-dc63-439d-a89f-ca9aebda54b7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:50.938117 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] The token cache is being invalidate because project 639dd1060d5043e982094872269651f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:50.938409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:50.939070 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:50.940890 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1200/2403] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:50 2022] DELETE /identity/v3/projects/639dd1060d5043e982094872269651f8 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:50.944837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2edbe5-91c8-4794-b070-6e3562dd542e service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:50.961947 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2edbe5-91c8-4794-b070-6e3562dd542e service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:51.019759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2de27d-6e72-4601-8d4b-ddae428df3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:51.020668 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2de27d-6e72-4601-8d4b-ddae428df3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:51.020947 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2de27d-6e72-4601-8d4b-ddae428df3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:51.021222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2de27d-6e72-4601-8d4b-ddae428df3d1 admin admin] PATH_INFO: `/v3/users/39056fc103c445ad823b95eac35b6356` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:51.022423 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2de27d-6e72-4601-8d4b-ddae428df3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39056fc103c445ad823b95eac35b6356)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:51.024366 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2de27d-6e72-4601-8d4b-ddae428df3d1 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:51.300153 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ba2de27d-6e72-4601-8d4b-ddae428df3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39056fc103c445ad823b95eac35b6356'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:51.300825 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ba2de27d-6e72-4601-8d4b-ddae428df3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39056fc103c445ad823b95eac35b6356'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:51.310856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ba2de27d-6e72-4601-8d4b-ddae428df3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39056fc103c445ad823b95eac35b6356'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:51.411085 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ba2de27d-6e72-4601-8d4b-ddae428df3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39056fc103c445ad823b95eac35b6356'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:51.411748 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ba2de27d-6e72-4601-8d4b-ddae428df3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39056fc103c445ad823b95eac35b6356'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:51.413139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:51.431022 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1204/2404] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:40:50 2022] DELETE /identity/v3/users/39056fc103c445ad823b95eac35b6356 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:51.964112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:51.966697 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:51.985923 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20dec6-dc63-439d-a89f-ca9aebda54b7 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:51.992669 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:52.050703 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782efdf-41ad-4c66-baef-9e78a97de065 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.051277 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782efdf-41ad-4c66-baef-9e78a97de065 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.052448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782efdf-41ad-4c66-baef-9e78a97de065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:52.077318 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:52.077956 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.078338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.078614 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] PATH_INFO: `/v3/projects/31ce763de2554b70b62a624e8182ce97` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:52.079781 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ce763de2554b70b62a624e8182ce97)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:52.082611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:52.091055 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-0782efdf-41ad-4c66-baef-9e78a97de065 None None] MFA Rules not processed for user `4770ff701ece49d894cc910e40b32733`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:52.093043 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0782efdf-41ad-4c66-baef-9e78a97de065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:52.176642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1205/2405] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:40:52.187759 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782efdf-41ad-4c66-baef-9e78a97de065 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:52.189438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0782efdf-41ad-4c66-baef-9e78a97de065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:52.216241 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ce763de2554b70b62a624e8182ce97', 'request_id': 'req-f191a231-1e82-4937-936f-d16a419bd749'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:52.223543 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ce763de2554b70b62a624e8182ce97', 'request_id': 'req-f191a231-1e82-4937-936f-d16a419bd749'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:52.263252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782efdf-41ad-4c66-baef-9e78a97de065 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:52.302110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ce763de2554b70b62a624e8182ce97', 'request_id': 'req-f191a231-1e82-4937-936f-d16a419bd749'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:52.317122 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be7ef2-167a-497e-8b3b-fe024d2525c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:52.320353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7ef2-167a-497e-8b3b-fe024d2525c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.320353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7ef2-167a-497e-8b3b-fe024d2525c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.320353 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be7ef2-167a-497e-8b3b-fe024d2525c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:52.335967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] The token cache is being invalidate because project 31ce763de2554b70b62a624e8182ce97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:40:52.336290 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:52.337008 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-f191a231-1e82-4937-936f-d16a419bd749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:40:52.339198 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1201/2406] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:40:51 2022] DELETE /identity/v3/projects/31ce763de2554b70b62a624e8182ce97 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:52.348602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c41427-888c-4d26-83e7-438f795a7450 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.348602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c41427-888c-4d26-83e7-438f795a7450 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.348602 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c41427-888c-4d26-83e7-438f795a7450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:52.365290 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c1c41427-888c-4d26-83e7-438f795a7450 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:52.366437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c1c41427-888c-4d26-83e7-438f795a7450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:52.366921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:52.383402 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be7ef2-167a-497e-8b3b-fe024d2525c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:52.385659 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be7ef2-167a-497e-8b3b-fe024d2525c1 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:52.396784 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1206/2407] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:40:52.426558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be7ef2-167a-497e-8b3b-fe024d2525c1 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:52.428002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-29be7ef2-167a-497e-8b3b-fe024d2525c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:52.443455 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1202/2408] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:52.456598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8300ced-edb9-4b80-8499-4a2b3d56589d None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.456891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8300ced-edb9-4b80-8499-4a2b3d56589d None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.457161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8300ced-edb9-4b80-8499-4a2b3d56589d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:52.482313 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-a8300ced-edb9-4b80-8499-4a2b3d56589d None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:52.484217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-a8300ced-edb9-4b80-8499-4a2b3d56589d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:52.500007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1203/2409] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:40:52.503800 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be7ef2-167a-497e-8b3b-fe024d2525c1 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:52.504180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8300ced-edb9-4b80-8499-4a2b3d56589d None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:52.507699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed7ac4-12c3-461a-bec6-2e4ed4c3c8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:52.508311 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed7ac4-12c3-461a-bec6-2e4ed4c3c8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.508593 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed7ac4-12c3-461a-bec6-2e4ed4c3c8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.508868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed7ac4-12c3-461a-bec6-2e4ed4c3c8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:52.511212 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-4bed7ac4-12c3-461a-bec6-2e4ed4c3c8d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:52.521468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8300ced-edb9-4b80-8499-4a2b3d56589d None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:52.524999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed7ac4-12c3-461a-bec6-2e4ed4c3c8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:52.529491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed7ac4-12c3-461a-bec6-2e4ed4c3c8d0 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:52.529631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398db3a-9b2b-429c-bbb5-0e9ccc91e193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:52.529631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398db3a-9b2b-429c-bbb5-0e9ccc91e193 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.529631 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398db3a-9b2b-429c-bbb5-0e9ccc91e193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.529950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398db3a-9b2b-429c-bbb5-0e9ccc91e193 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:52.530496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398db3a-9b2b-429c-bbb5-0e9ccc91e193 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:40:52.530795 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398db3a-9b2b-429c-bbb5-0e9ccc91e193 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:52.532389 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398db3a-9b2b-429c-bbb5-0e9ccc91e193 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:52.538953 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1207/2410] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:40:52.550324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1204/2411] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:40:52.555184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed7ac4-12c3-461a-bec6-2e4ed4c3c8d0 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:52.565635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed7ac4-12c3-461a-bec6-2e4ed4c3c8d0 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:52.570601 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057f155-ab19-48de-9522-17ddfe7f76d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:52.571392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057f155-ab19-48de-9522-17ddfe7f76d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.571780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057f155-ab19-48de-9522-17ddfe7f76d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.572158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057f155-ab19-48de-9522-17ddfe7f76d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:52.573418 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6057f155-ab19-48de-9522-17ddfe7f76d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:52.575344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6057f155-ab19-48de-9522-17ddfe7f76d4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:52.688052 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1208/2412] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:40:52 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:40:52.692366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398db3a-9b2b-429c-bbb5-0e9ccc91e193 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:52.703088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398db3a-9b2b-429c-bbb5-0e9ccc91e193 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:52.761468 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642afa1-96a5-4607-88af-939c59106461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:52.762098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642afa1-96a5-4607-88af-939c59106461 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.762374 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642afa1-96a5-4607-88af-939c59106461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.762639 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642afa1-96a5-4607-88af-939c59106461 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:52.763293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642afa1-96a5-4607-88af-939c59106461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:52.765292 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642afa1-96a5-4607-88af-939c59106461 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:52.890134 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1205/2413] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:40:52 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:40:52.891265 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057f155-ab19-48de-9522-17ddfe7f76d4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:52.902164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6057f155-ab19-48de-9522-17ddfe7f76d4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:52.906988 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2366ac-1325-414c-8198-0d748d446f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:52.911847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2366ac-1325-414c-8198-0d748d446f76 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.911847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2366ac-1325-414c-8198-0d748d446f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.911847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2366ac-1325-414c-8198-0d748d446f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:52.911847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2366ac-1325-414c-8198-0d748d446f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:52.911847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2366ac-1325-414c-8198-0d748d446f76 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:52.924371 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1209/2414] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:40:52.928503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642afa1-96a5-4607-88af-939c59106461 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:52.941998 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:40:52.948059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642afa1-96a5-4607-88af-939c59106461 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:52.953007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013ce4a-2772-4fc9-9a9d-77a4c2b163fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:52.953628 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013ce4a-2772-4fc9-9a9d-77a4c2b163fd admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.953904 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013ce4a-2772-4fc9-9a9d-77a4c2b163fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.954203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013ce4a-2772-4fc9-9a9d-77a4c2b163fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:52.954852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013ce4a-2772-4fc9-9a9d-77a4c2b163fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:52.956466 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013ce4a-2772-4fc9-9a9d-77a4c2b163fd admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:52.972854 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1206/2415] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:40:52.975017 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2366ac-1325-414c-8198-0d748d446f76 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:52.988013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2366ac-1325-414c-8198-0d748d446f76 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:52.996202 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24df47-4d5c-45c4-9477-c1e37f569fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:52.998218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24df47-4d5c-45c4-9477-c1e37f569fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:52.998218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24df47-4d5c-45c4-9477-c1e37f569fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:52.998218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24df47-4d5c-45c4-9477-c1e37f569fcf admin admin] PATH_INFO: `/v3/projects/e9a2f37623a34e85ac89df451afcf5e5/users/02b0ae78396f449caccc0f59e87f87ae/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:53.013317 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24df47-4d5c-45c4-9477-c1e37f569fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9a2f37623a34e85ac89df451afcf5e5, user_id=02b0ae78396f449caccc0f59e87f87ae, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:53.016042 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24df47-4d5c-45c4-9477-c1e37f569fcf admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:53.107199 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1210/2416] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:40:52 2022] PUT /identity/v3/projects/e9a2f37623a34e85ac89df451afcf5e5/users/02b0ae78396f449caccc0f59e87f87ae/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:40:53.110335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d04a0-a114-475f-9e42-1c49ebf9cc51 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:53.110616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d04a0-a114-475f-9e42-1c49ebf9cc51 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:53.110887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d04a0-a114-475f-9e42-1c49ebf9cc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:53.139258 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-691d04a0-a114-475f-9e42-1c49ebf9cc51 None None] MFA Rules not processed for user `02b0ae78396f449caccc0f59e87f87ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:53.140623 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-691d04a0-a114-475f-9e42-1c49ebf9cc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:53.216109 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1207/2417] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:40:53.227442 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24df47-4d5c-45c4-9477-c1e37f569fcf admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:40:53.230030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7f24df47-4d5c-45c4-9477-c1e37f569fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:53.301730 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24df47-4d5c-45c4-9477-c1e37f569fcf admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:40:53.305435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50b24-1b19-476b-b1a0-68dcfb239145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:40:53.306075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50b24-1b19-476b-b1a0-68dcfb239145 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:53.306352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50b24-1b19-476b-b1a0-68dcfb239145 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:53.306621 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe50b24-1b19-476b-b1a0-68dcfb239145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:53.314979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe50b24-1b19-476b-b1a0-68dcfb239145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:40:53.316412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe50b24-1b19-476b-b1a0-68dcfb239145 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:40:53.325190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1211/2418] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:40:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:40:55.384028 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef220c6-fb8f-480b-9a47-f53009276ffc None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:40:55.384508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef220c6-fb8f-480b-9a47-f53009276ffc None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:40:55.388307 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef220c6-fb8f-480b-9a47-f53009276ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:40:55.409744 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-cef220c6-fb8f-480b-9a47-f53009276ffc None None] MFA Rules not processed for user `4770ff701ece49d894cc910e40b32733`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:40:55.411145 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-cef220c6-fb8f-480b-9a47-f53009276ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:40:55.480629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1208/2419] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:40:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:41:02.029024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50b24-1b19-476b-b1a0-68dcfb239145 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:41:02.030574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5fe50b24-1b19-476b-b1a0-68dcfb239145 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:41:02.073701 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:41:02.128542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe50b24-1b19-476b-b1a0-68dcfb239145 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:41:02.137175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:41:02.137175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:41:02.137175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:41:02.137175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] PATH_INFO: `/v3/users/a2e0fd0cc64e4323a021ed6a36943986` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:41:02.138740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e0fd0cc64e4323a021ed6a36943986)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:41:02.141185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:41:02.373527 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e0fd0cc64e4323a021ed6a36943986'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:02.374172 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e0fd0cc64e4323a021ed6a36943986'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:02.381185 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e0fd0cc64e4323a021ed6a36943986'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:02.450592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e0fd0cc64e4323a021ed6a36943986'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:02.451240 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e0fd0cc64e4323a021ed6a36943986'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:02.469405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1212/2420] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:41:02 2022] DELETE /identity/v3/users/a2e0fd0cc64e4323a021ed6a36943986 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:41:02.801312 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef220c6-fb8f-480b-9a47-f53009276ffc None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:41:02.817088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef220c6-fb8f-480b-9a47-f53009276ffc None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:41:02.866016 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:41:02.893160 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:41:02.893791 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:41:02.894544 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:41:02.894544 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] PATH_INFO: `/v3/projects/4314958c672547cba046eb6c3af918fe` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:41:02.895459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4314958c672547cba046eb6c3af918fe)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:41:02.897363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:41:03.076038 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4314958c672547cba046eb6c3af918fe', 'request_id': 'req-22896d06-df54-47e0-8e0c-ce3f646a1e7c'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:03.082297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4314958c672547cba046eb6c3af918fe', 'request_id': 'req-22896d06-df54-47e0-8e0c-ce3f646a1e7c'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:03.182708 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4314958c672547cba046eb6c3af918fe', 'request_id': 'req-22896d06-df54-47e0-8e0c-ce3f646a1e7c'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:03.193759 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] The token cache is being invalidate because project 4314958c672547cba046eb6c3af918fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:41:03.194059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:03.194728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:03.196192 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1209/2421] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:41:02 2022] DELETE /identity/v3/projects/4314958c672547cba046eb6c3af918fe => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:41:03.201020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:41:03.202469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:41:03.215917 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e3bd6-dcef-45fd-977b-0b48b40265da admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:41:03.272200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:41:03.272744 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:41:03.273025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:41:03.273297 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] PATH_INFO: `/v3/users/8e7e1f922aaa448098051f7bb8ceaca5` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:41:03.274478 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e7e1f922aaa448098051f7bb8ceaca5)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:41:03.276457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:41:03.476977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e7e1f922aaa448098051f7bb8ceaca5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:03.477626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e7e1f922aaa448098051f7bb8ceaca5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:03.480455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:41:03.485840 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e7e1f922aaa448098051f7bb8ceaca5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:03.542627 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e7e1f922aaa448098051f7bb8ceaca5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:03.543179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e7e1f922aaa448098051f7bb8ceaca5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:03.561059 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1213/2422] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:41:03 2022] DELETE /identity/v3/users/8e7e1f922aaa448098051f7bb8ceaca5 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:41:03.795783 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:41:03.797222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:41:03.816688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22896d06-df54-47e0-8e0c-ce3f646a1e7c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:41:03.873126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:41:03.873750 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:41:03.874028 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:41:03.874326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] PATH_INFO: `/v3/projects/c07b4c56c1c34f29b7c34edb664bae27` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:41:03.875451 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c07b4c56c1c34f29b7c34edb664bae27)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:41:03.877616 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:41:04.006739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c07b4c56c1c34f29b7c34edb664bae27', 'request_id': 'req-b4793e7f-e288-4c93-aff1-57ff6876bc15'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:04.013585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c07b4c56c1c34f29b7c34edb664bae27', 'request_id': 'req-b4793e7f-e288-4c93-aff1-57ff6876bc15'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:04.073590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c07b4c56c1c34f29b7c34edb664bae27', 'request_id': 'req-b4793e7f-e288-4c93-aff1-57ff6876bc15'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:04.078314 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:41:04.086599 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] The token cache is being invalidate because project c07b4c56c1c34f29b7c34edb664bae27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:41:04.089504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:04.089504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:04.089880 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1210/2423] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:41:03 2022] DELETE /identity/v3/projects/c07b4c56c1c34f29b7c34edb664bae27 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:41:04.323182 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:41:04.324647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:41:04.382792 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c4980-605b-4c05-aff4-3a593f1f4370 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:41:04.429742 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363479da-a08a-467d-ae89-5c8190c27a9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:41:04.430452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363479da-a08a-467d-ae89-5c8190c27a9a service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:41:04.430733 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363479da-a08a-467d-ae89-5c8190c27a9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:41:04.431006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363479da-a08a-467d-ae89-5c8190c27a9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:41:04.432726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-363479da-a08a-467d-ae89-5c8190c27a9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:41:04.494145 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363479da-a08a-467d-ae89-5c8190c27a9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:41:04.496071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363479da-a08a-467d-ae89-5c8190c27a9a service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:41:04.505913 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1214/2424] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:41:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:41:06.096426 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:41:06.109283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:41:06.181646 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4793e7f-e288-4c93-aff1-57ff6876bc15 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:41:06.188491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:41:06.188491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:41:06.188491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:41:06.188491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] PATH_INFO: `/v3/users/02b0ae78396f449caccc0f59e87f87ae` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:41:06.188993 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b0ae78396f449caccc0f59e87f87ae)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:41:06.190975 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:41:06.420565 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b0ae78396f449caccc0f59e87f87ae'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:06.421530 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b0ae78396f449caccc0f59e87f87ae'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:06.491111 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b0ae78396f449caccc0f59e87f87ae'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:06.491762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b0ae78396f449caccc0f59e87f87ae'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:06.502368 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b0ae78396f449caccc0f59e87f87ae'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:06.521064 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1211/2425] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:41:06 2022] DELETE /identity/v3/users/02b0ae78396f449caccc0f59e87f87ae => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:41:06.526096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa80d56-34e3-4c3e-a344-f4e4a5919c18 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:41:06.526591 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa80d56-34e3-4c3e-a344-f4e4a5919c18 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:41:06.527935 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa80d56-34e3-4c3e-a344-f4e4a5919c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:41:06.546837 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-cfa80d56-34e3-4c3e-a344-f4e4a5919c18 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:41:06.547979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cfa80d56-34e3-4c3e-a344-f4e4a5919c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:41:06.603496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:41:06.641335 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1215/2426] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:41:06.666754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:41:06.668243 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:41:06.735666 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ff109-044e-470a-9240-e7d48e2e40fc admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:41:06.740282 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576067d-6243-4d98-a19b-675c83c6d341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:41:06.741044 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576067d-6243-4d98-a19b-675c83c6d341 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:41:06.741320 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576067d-6243-4d98-a19b-675c83c6d341 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:41:06.742093 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576067d-6243-4d98-a19b-675c83c6d341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:41:06.753888 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576067d-6243-4d98-a19b-675c83c6d341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:41:06.755465 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e576067d-6243-4d98-a19b-675c83c6d341 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:41:06.767555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1212/2427] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:41:07.384413 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa80d56-34e3-4c3e-a344-f4e4a5919c18 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:41:07.393042 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-cfa80d56-34e3-4c3e-a344-f4e4a5919c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:41:07.400569 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa80d56-34e3-4c3e-a344-f4e4a5919c18 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:41:07.405439 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:41:07.406288 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:41:07.406448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:41:07.406660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] PATH_INFO: `/v3/projects/e9a2f37623a34e85ac89df451afcf5e5` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:41:07.407778 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9a2f37623a34e85ac89df451afcf5e5)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:41:07.409921 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:41:07.510958 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9a2f37623a34e85ac89df451afcf5e5', 'request_id': 'req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:07.517989 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9a2f37623a34e85ac89df451afcf5e5', 'request_id': 'req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:07.599130 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9a2f37623a34e85ac89df451afcf5e5', 'request_id': 'req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:07.611087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] The token cache is being invalidate because project e9a2f37623a34e85ac89df451afcf5e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:41:07.611393 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:07.612104 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-d0ee1052-d6c3-42fb-ba40-cbe0f36a611f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:41:07.613996 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1216/2428] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:41:07 2022] DELETE /identity/v3/projects/e9a2f37623a34e85ac89df451afcf5e5 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:41:16.761775 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:DELETE=11,keystone:SELECT=81,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:41:17.607810 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:41:33.739279 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a4a11-c771-40d8-88f6-316251fbf955 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:41:33.747587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a4a11-c771-40d8-88f6-316251fbf955 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:41:33.747587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a4a11-c771-40d8-88f6-316251fbf955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:41:33.758873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-d01a4a11-c771-40d8-88f6-316251fbf955 None None] MFA Rules not processed for user `1d16e478c25946a8818641b7d5ea2e15`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:41:33.760055 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d01a4a11-c771-40d8-88f6-316251fbf955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:41:33.813371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1213/2429] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:41:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:41:43.806852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=25 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:42:09.498517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac920d4-c758-40ba-9396-d2f49cebe780 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:42:09.502024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac920d4-c758-40ba-9396-d2f49cebe780 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:42:09.502024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac920d4-c758-40ba-9396-d2f49cebe780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:42:09.518870 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-0ac920d4-c758-40ba-9396-d2f49cebe780 None None] MFA Rules not processed for user `4770ff701ece49d894cc910e40b32733`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:42:09.520062 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0ac920d4-c758-40ba-9396-d2f49cebe780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:42:09.573969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1217/2430] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:42:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:42:16.190734 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a4a11-c771-40d8-88f6-316251fbf955 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:42:16.192233 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d01a4a11-c771-40d8-88f6-316251fbf955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:42:16.249748 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a4a11-c771-40d8-88f6-316251fbf955 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:42:16.253550 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5733897-a581-448b-9815-d787b97f7a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:42:16.254184 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5733897-a581-448b-9815-d787b97f7a75 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:42:16.254465 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5733897-a581-448b-9815-d787b97f7a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:42:16.254737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5733897-a581-448b-9815-d787b97f7a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:42:16.256463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-d5733897-a581-448b-9815-d787b97f7a75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:42:16.336531 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5733897-a581-448b-9815-d787b97f7a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:42:16.337957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5733897-a581-448b-9815-d787b97f7a75 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:42:16.357707 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1214/2431] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:42:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:42:19.566912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=25 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:42:19.873125 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac920d4-c758-40ba-9396-d2f49cebe780 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:42:19.874536 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0ac920d4-c758-40ba-9396-d2f49cebe780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:42:19.886940 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac920d4-c758-40ba-9396-d2f49cebe780 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:42:19.890404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ac63d-229a-4d19-ba5c-2a44c487dfe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:42:19.891101 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ac63d-229a-4d19-ba5c-2a44c487dfe4 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:42:19.891256 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ac63d-229a-4d19-ba5c-2a44c487dfe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:42:19.891523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ac63d-229a-4d19-ba5c-2a44c487dfe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:42:19.893117 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e22ac63d-229a-4d19-ba5c-2a44c487dfe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:42:19.905839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ac63d-229a-4d19-ba5c-2a44c487dfe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:42:19.907273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ac63d-229a-4d19-ba5c-2a44c487dfe4 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:42:19.917852 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1218/2432] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:42:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:42:26.355157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=63 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:42:29.913162 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=6 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:42:46.224042 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73b95a-0472-47a6-9e9e-9d5ff8655813 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:42:46.230058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73b95a-0472-47a6-9e9e-9d5ff8655813 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:42:46.230058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73b95a-0472-47a6-9e9e-9d5ff8655813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:42:46.273925 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-4c73b95a-0472-47a6-9e9e-9d5ff8655813 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:42:46.275082 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4c73b95a-0472-47a6-9e9e-9d5ff8655813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:42:46.338271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1215/2433] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:42:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:42:46.494578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ac63d-229a-4d19-ba5c-2a44c487dfe4 service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:42:46.496109 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e22ac63d-229a-4d19-ba5c-2a44c487dfe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:42:46.588629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ac63d-229a-4d19-ba5c-2a44c487dfe4 service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:42:46.593794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5350980-3f65-4a02-be30-27b0bcd41ac0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:42:46.594451 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5350980-3f65-4a02-be30-27b0bcd41ac0 service swift] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:42:46.594732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5350980-3f65-4a02-be30-27b0bcd41ac0 service swift] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:42:46.595010 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5350980-3f65-4a02-be30-27b0bcd41ac0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:42:46.602904 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5350980-3f65-4a02-be30-27b0bcd41ac0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:42:46.604477 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5350980-3f65-4a02-be30-27b0bcd41ac0 service swift] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:42:46.612448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1219/2434] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:42:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:42:56.335450 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=40 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:42:56.609281 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=41 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:43:02.617078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73b95a-0472-47a6-9e9e-9d5ff8655813 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:02.618765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4c73b95a-0472-47a6-9e9e-9d5ff8655813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:02.635896 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73b95a-0472-47a6-9e9e-9d5ff8655813 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:02.642884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa991f-e20f-4a9f-860f-6f6305dac5f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:02.643676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa991f-e20f-4a9f-860f-6f6305dac5f6 service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:02.644028 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa991f-e20f-4a9f-860f-6f6305dac5f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:02.644375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa991f-e20f-4a9f-860f-6f6305dac5f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:02.646465 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4afa991f-e20f-4a9f-860f-6f6305dac5f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:02.660249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa991f-e20f-4a9f-860f-6f6305dac5f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:02.664661 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa991f-e20f-4a9f-860f-6f6305dac5f6 service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:02.674202 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1216/2435] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:43:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:43:03.149532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5350980-3f65-4a02-be30-27b0bcd41ac0 service swift] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:03.161829 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5350980-3f65-4a02-be30-27b0bcd41ac0 service swift] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:03.176127 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95e26f-b4b6-4ece-b8d6-a34e7154ba09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:03.176855 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95e26f-b4b6-4ece-b8d6-a34e7154ba09 service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:03.177231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95e26f-b4b6-4ece-b8d6-a34e7154ba09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:03.177722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95e26f-b4b6-4ece-b8d6-a34e7154ba09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:03.179626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ff95e26f-b4b6-4ece-b8d6-a34e7154ba09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:03.193565 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95e26f-b4b6-4ece-b8d6-a34e7154ba09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:03.195121 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95e26f-b4b6-4ece-b8d6-a34e7154ba09 service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:03.205614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1220/2436] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:43:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:43:06.285196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5393c5-63ad-46ae-8a9c-d421b6effe51 None None] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:06.285789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5393c5-63ad-46ae-8a9c-d421b6effe51 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:06.285789 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5393c5-63ad-46ae-8a9c-d421b6effe51 None None] PATH_INFO: `/` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:06.287153 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1217/2437] 10.222.0.20 () {54 vars in 921 bytes} [Wed Oct 19 02:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 19 02:43:06.291822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c813f-5673-4ebb-b775-79a05bec9081 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:06.292298 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c813f-5673-4ebb-b775-79a05bec9081 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:06.292412 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c813f-5673-4ebb-b775-79a05bec9081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:06.330288 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-502c813f-5673-4ebb-b775-79a05bec9081 None None] MFA Rules not processed for user `0abf52399a344157ac4e0f9a8212164e`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:43:06.331489 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-502c813f-5673-4ebb-b775-79a05bec9081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:06.340946 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1221/2438] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Oct 19 02:43:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:43:06.353240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5393c5-63ad-46ae-8a9c-d421b6effe51 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:06.355057 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-bc5393c5-63ad-46ae-8a9c-d421b6effe51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:06.413675 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5393c5-63ad-46ae-8a9c-d421b6effe51 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:06.417119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2e9d2-18c1-4738-ac22-1d5fb7234d36 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '286cfcbb8334421f87c0d426aa52a30a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:06.417718 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2e9d2-18c1-4738-ac22-1d5fb7234d36 service glance] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:06.417997 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2e9d2-18c1-4738-ac22-1d5fb7234d36 service glance] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:06.418300 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2e9d2-18c1-4738-ac22-1d5fb7234d36 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:06.428180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2e9d2-18c1-4738-ac22-1d5fb7234d36 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:06.429611 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2e9d2-18c1-4738-ac22-1d5fb7234d36 service glance] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:06.440018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1218/2439] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:43:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:43:10.667578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e23728-5074-4a0a-bf3e-c4ff9240528c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:10.668084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e23728-5074-4a0a-bf3e-c4ff9240528c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:10.668189 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e23728-5074-4a0a-bf3e-c4ff9240528c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:10.687892 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-62e23728-5074-4a0a-bf3e-c4ff9240528c None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:43:10.689228 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-62e23728-5074-4a0a-bf3e-c4ff9240528c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:10.703799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1222/2440] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Oct 19 02:43:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:43:10.709173 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2e9d2-18c1-4738-ac22-1d5fb7234d36 service glance] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:10.722256 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2e9d2-18c1-4738-ac22-1d5fb7234d36 service glance] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:10.727263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86321bf7-ac91-40bc-901d-6b95959b9370 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:10.727847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86321bf7-ac91-40bc-901d-6b95959b9370 service swift] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:10.728119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86321bf7-ac91-40bc-901d-6b95959b9370 service swift] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:10.728387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86321bf7-ac91-40bc-901d-6b95959b9370 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:10.736271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86321bf7-ac91-40bc-901d-6b95959b9370 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:10.737823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86321bf7-ac91-40bc-901d-6b95959b9370 service swift] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:10.745689 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1219/2441] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:43:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 19 02:43:13.635388 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e23728-5074-4a0a-bf3e-c4ff9240528c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:13.646967 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e23728-5074-4a0a-bf3e-c4ff9240528c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:13.654734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683b098-c149-48b5-b4f8-ec3d29dcc65d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:13.657226 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683b098-c149-48b5-b4f8-ec3d29dcc65d service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:13.657670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683b098-c149-48b5-b4f8-ec3d29dcc65d service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:13.659188 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f683b098-c149-48b5-b4f8-ec3d29dcc65d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:13.662436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f683b098-c149-48b5-b4f8-ec3d29dcc65d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:13.745872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683b098-c149-48b5-b4f8-ec3d29dcc65d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:13.747432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f683b098-c149-48b5-b4f8-ec3d29dcc65d service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:13.758689 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1223/2442] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:43:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:43:18.668387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86321bf7-ac91-40bc-901d-6b95959b9370 service swift] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:18.670657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-86321bf7-ac91-40bc-901d-6b95959b9370 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:18.768752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86321bf7-ac91-40bc-901d-6b95959b9370 service swift] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:18.777381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f01f61-27c2-414b-a217-05a6e9fe66f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:18.778434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f01f61-27c2-414b-a217-05a6e9fe66f7 service placement] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:18.778970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f01f61-27c2-414b-a217-05a6e9fe66f7 service placement] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:18.779383 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f01f61-27c2-414b-a217-05a6e9fe66f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:18.781992 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-99f01f61-27c2-414b-a217-05a6e9fe66f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:18.799306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f01f61-27c2-414b-a217-05a6e9fe66f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:18.800926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f01f61-27c2-414b-a217-05a6e9fe66f7 service placement] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:18.820001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1220/2443] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:43:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:43:21.530822 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683b098-c149-48b5-b4f8-ec3d29dcc65d service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:21.532328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-f683b098-c149-48b5-b4f8-ec3d29dcc65d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:21.545193 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f683b098-c149-48b5-b4f8-ec3d29dcc65d service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:21.552223 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:21.552965 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:21.553318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:21.553589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] PATH_INFO: `/v3/users/4770ff701ece49d894cc910e40b32733` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:21.554839 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4770ff701ece49d894cc910e40b32733)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:21.556997 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:21.767609 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4770ff701ece49d894cc910e40b32733'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:21.768262 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4770ff701ece49d894cc910e40b32733'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:21.777558 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4770ff701ece49d894cc910e40b32733'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:21.886204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f01f61-27c2-414b-a217-05a6e9fe66f7 service placement] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:21.887671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-99f01f61-27c2-414b-a217-05a6e9fe66f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:21.903974 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f01f61-27c2-414b-a217-05a6e9fe66f7 service placement] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:21.910189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d74b6-901e-4bf8-955c-d14b90aac90f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0abf52399a344157ac4e0f9a8212164e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:21.910801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d74b6-901e-4bf8-955c-d14b90aac90f service nova] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:21.911079 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d74b6-901e-4bf8-955c-d14b90aac90f service nova] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:21.911353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d74b6-901e-4bf8-955c-d14b90aac90f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:21.913070 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ca6d74b6-901e-4bf8-955c-d14b90aac90f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:21.932018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d74b6-901e-4bf8-955c-d14b90aac90f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:21.933622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d74b6-901e-4bf8-955c-d14b90aac90f service nova] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:21.943442 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1221/2444] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:43:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:43:21.945170 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4770ff701ece49d894cc910e40b32733'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:21.948102 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4770ff701ece49d894cc910e40b32733'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:21.983938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1224/2445] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:43:21 2022] DELETE /identity/v3/users/4770ff701ece49d894cc910e40b32733 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:43:22.692371 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d74b6-901e-4bf8-955c-d14b90aac90f service nova] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:22.693897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ca6d74b6-901e-4bf8-955c-d14b90aac90f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:22.709301 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d74b6-901e-4bf8-955c-d14b90aac90f service nova] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:22.759092 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:43:22.768503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:22.771816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:22.771816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:22.771816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] PATH_INFO: `/v3/projects/9fb22a6e7d3244b2b1b3b5b1a3b9a963` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:22.771816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb22a6e7d3244b2b1b3b5b1a3b9a963)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:22.772773 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:22.815398 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:22.816919 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:22.832074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67fed2-3be5-4eed-b50c-8a1f96906fa1 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:22.835421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:22.836094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:22.836310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:22.836581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] PATH_INFO: `/v3/users/0bd6670e1a0d46b088c13e82e31dc6da` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:22.837730 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bd6670e1a0d46b088c13e82e31dc6da)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:22.839677 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:22.862694 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:43:22.954504 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb22a6e7d3244b2b1b3b5b1a3b9a963', 'request_id': 'req-2e8b621e-0b54-4a88-8098-4d93c3faf09c'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:22.963071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb22a6e7d3244b2b1b3b5b1a3b9a963', 'request_id': 'req-2e8b621e-0b54-4a88-8098-4d93c3faf09c'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:23.867457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb22a6e7d3244b2b1b3b5b1a3b9a963', 'request_id': 'req-2e8b621e-0b54-4a88-8098-4d93c3faf09c'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:23.879169 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] The token cache is being invalidate because project 9fb22a6e7d3244b2b1b3b5b1a3b9a963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:43:23.901457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:23.901457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-2e8b621e-0b54-4a88-8098-4d93c3faf09c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:23.901739 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bd6670e1a0d46b088c13e82e31dc6da'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:23.902227 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1222/2446] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:43:22 2022] DELETE /identity/v3/projects/9fb22a6e7d3244b2b1b3b5b1a3b9a963 => generated 0 bytes in 1190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:43:23.902327 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bd6670e1a0d46b088c13e82e31dc6da'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:23.902436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72890045-599c-4e26-97f4-2a290e15b16a None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:23.902436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72890045-599c-4e26-97f4-2a290e15b16a None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:23.902436 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72890045-599c-4e26-97f4-2a290e15b16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:23.902720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bd6670e1a0d46b088c13e82e31dc6da'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:23.906163 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-72890045-599c-4e26-97f4-2a290e15b16a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:43:23.907899 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-72890045-599c-4e26-97f4-2a290e15b16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:23.965529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1223/2447] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:43:23.973605 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1066d-8dab-4eb0-a689-f5d6a0e2f67c None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:23.973915 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1066d-8dab-4eb0-a689-f5d6a0e2f67c None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:23.974214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff1066d-8dab-4eb0-a689-f5d6a0e2f67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:23.994534 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-1ff1066d-8dab-4eb0-a689-f5d6a0e2f67c None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:43:23.995703 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1ff1066d-8dab-4eb0-a689-f5d6a0e2f67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:24.004242 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1224/2448] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:43:24.009435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1066d-8dab-4eb0-a689-f5d6a0e2f67c None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:24.018251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff1066d-8dab-4eb0-a689-f5d6a0e2f67c None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:24.022855 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e3bd4-6493-4971-a57d-fa184bb72dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:24.023417 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e3bd4-6493-4971-a57d-fa184bb72dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:24.023695 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e3bd4-6493-4971-a57d-fa184bb72dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:24.023964 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e3bd4-6493-4971-a57d-fa184bb72dd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:24.024589 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e3bd4-6493-4971-a57d-fa184bb72dd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:43:24.024888 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e3bd4-6493-4971-a57d-fa184bb72dd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:24.026567 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e3bd4-6493-4971-a57d-fa184bb72dd6 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:24.027415 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bd6670e1a0d46b088c13e82e31dc6da'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:24.028030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bd6670e1a0d46b088c13e82e31dc6da'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:24.039187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1225/2449] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 19 02:43:24.043490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e3bd4-6493-4971-a57d-fa184bb72dd6 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:24.046734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1225/2450] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:43:22 2022] DELETE /identity/v3/users/0bd6670e1a0d46b088c13e82e31dc6da => generated 0 bytes in 1232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:43:24.055472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e3bd4-6493-4971-a57d-fa184bb72dd6 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:24.119549 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c749f-7268-487f-aa94-f8b25c590857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:24.120081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c749f-7268-487f-aa94-f8b25c590857 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:24.120366 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c749f-7268-487f-aa94-f8b25c590857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:24.120628 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c749f-7268-487f-aa94-f8b25c590857 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:24.121257 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c749f-7268-487f-aa94-f8b25c590857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:24.123189 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c749f-7268-487f-aa94-f8b25c590857 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:24.139161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:43:24.210663 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1226/2451] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:43:24 2022] POST /identity/v3/projects => generated 391 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:43:24.214455 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:24.223561 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1affc8eb-6e9d-4bc5-a976-4a1957c6b8cd admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:24.280137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4fdfb-5bdb-4fbc-9146-453d4f8c0d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:24.280746 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4fdfb-5bdb-4fbc-9146-453d4f8c0d1d admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:24.281030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4fdfb-5bdb-4fbc-9146-453d4f8c0d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:24.281301 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4fdfb-5bdb-4fbc-9146-453d4f8c0d1d admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:24.281943 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4fdfb-5bdb-4fbc-9146-453d4f8c0d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:24.284566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded4fdfb-5bdb-4fbc-9146-453d4f8c0d1d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:24.332209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c749f-7268-487f-aa94-f8b25c590857 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:24.334851 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-503c749f-7268-487f-aa94-f8b25c590857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:24.347051 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c749f-7268-487f-aa94-f8b25c590857 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:24.351977 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:24.352630 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:24.352905 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:24.353206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] PATH_INFO: `/v3/projects/2e98a99d79ef470d940dd16869a9083f` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:24.354412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e98a99d79ef470d940dd16869a9083f)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:24.356338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:24.418853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1226/2452] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:43:24 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:43:24.419979 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4fdfb-5bdb-4fbc-9146-453d4f8c0d1d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:24.430796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded4fdfb-5bdb-4fbc-9146-453d4f8c0d1d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:24.435954 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc6185-5676-41a1-aa0c-46bbd98fe9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:24.436557 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc6185-5676-41a1-aa0c-46bbd98fe9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:24.437002 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc6185-5676-41a1-aa0c-46bbd98fe9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:24.437273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc6185-5676-41a1-aa0c-46bbd98fe9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:24.437916 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc6185-5676-41a1-aa0c-46bbd98fe9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:24.439541 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc6185-5676-41a1-aa0c-46bbd98fe9c7 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:24.450351 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1227/2453] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:43:24.454707 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc6185-5676-41a1-aa0c-46bbd98fe9c7 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:24.465168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc6185-5676-41a1-aa0c-46bbd98fe9c7 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:24.471050 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2764a-65e0-424c-9a79-0cebc9bb6fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:24.471654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2764a-65e0-424c-9a79-0cebc9bb6fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:24.471930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2764a-65e0-424c-9a79-0cebc9bb6fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:24.472206 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2764a-65e0-424c-9a79-0cebc9bb6fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:24.472851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2764a-65e0-424c-9a79-0cebc9bb6fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:24.474538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2764a-65e0-424c-9a79-0cebc9bb6fc5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:24.485681 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1228/2454] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:43:24.489985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2764a-65e0-424c-9a79-0cebc9bb6fc5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:24.498036 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e98a99d79ef470d940dd16869a9083f', 'request_id': 'req-714d947c-660d-4b11-b288-9d2393e8b319'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:24.500061 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2764a-65e0-424c-9a79-0cebc9bb6fc5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:24.505588 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e98a99d79ef470d940dd16869a9083f', 'request_id': 'req-714d947c-660d-4b11-b288-9d2393e8b319'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:24.558337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158e0ea-f032-4dc0-8145-40bbffe1e129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:24.558990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158e0ea-f032-4dc0-8145-40bbffe1e129 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:24.559263 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158e0ea-f032-4dc0-8145-40bbffe1e129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:24.559531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158e0ea-f032-4dc0-8145-40bbffe1e129 admin admin] PATH_INFO: `/v3/projects/338befa62e384b6ab7ca5e66af885c06/users/bd40f521bebc4f85a1409e9f9a8936db/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:24.570517 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:43:24.574244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158e0ea-f032-4dc0-8145-40bbffe1e129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=338befa62e384b6ab7ca5e66af885c06, user_id=bd40f521bebc4f85a1409e9f9a8936db, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:24.576894 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158e0ea-f032-4dc0-8145-40bbffe1e129 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:24.587204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e98a99d79ef470d940dd16869a9083f', 'request_id': 'req-714d947c-660d-4b11-b288-9d2393e8b319'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:24.597873 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] The token cache is being invalidate because project 2e98a99d79ef470d940dd16869a9083f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:43:24.598187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:24.598818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:24.600335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1227/2455] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:43:24 2022] DELETE /identity/v3/projects/2e98a99d79ef470d940dd16869a9083f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:43:24.604119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:24.605404 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:24.617658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d947c-660d-4b11-b288-9d2393e8b319 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:24.622462 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfb478-e064-4773-b8d4-dba13853a2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:24.626518 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb478-e064-4773-b8d4-dba13853a2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:24.626518 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb478-e064-4773-b8d4-dba13853a2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:24.626518 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb478-e064-4773-b8d4-dba13853a2d4 admin admin] PATH_INFO: `/v3/users/eb62cc422387437dbef9bd444192cf06` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:24.626518 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfb478-e064-4773-b8d4-dba13853a2d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb62cc422387437dbef9bd444192cf06)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:24.627480 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfb478-e064-4773-b8d4-dba13853a2d4 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:24.684602 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1229/2456] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:43:24 2022] PUT /identity/v3/projects/338befa62e384b6ab7ca5e66af885c06/users/bd40f521bebc4f85a1409e9f9a8936db/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:43:24.694607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e5369-0827-4c79-9215-74fa8b769d73 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:24.694890 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e5369-0827-4c79-9215-74fa8b769d73 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:24.695192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e5369-0827-4c79-9215-74fa8b769d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:24.723610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-1f7e5369-0827-4c79-9215-74fa8b769d73 None None] MFA Rules not processed for user `bd40f521bebc4f85a1409e9f9a8936db`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:43:24.724803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1f7e5369-0827-4c79-9215-74fa8b769d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:24.792799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1230/2457] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:43:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:43:24.803647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e5369-0827-4c79-9215-74fa8b769d73 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:24.804975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-1f7e5369-0827-4c79-9215-74fa8b769d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:24.864024 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e5369-0827-4c79-9215-74fa8b769d73 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:24.867685 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718b7ae-55d2-41a3-ae35-55c4d6833c26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:24.868283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718b7ae-55d2-41a3-ae35-55c4d6833c26 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:24.868560 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718b7ae-55d2-41a3-ae35-55c4d6833c26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:24.868831 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718b7ae-55d2-41a3-ae35-55c4d6833c26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:24.877672 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718b7ae-55d2-41a3-ae35-55c4d6833c26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:24.879383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718b7ae-55d2-41a3-ae35-55c4d6833c26 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:24.888282 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1231/2458] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:43:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 19 02:43:24.897844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8ccfb478-e064-4773-b8d4-dba13853a2d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb62cc422387437dbef9bd444192cf06'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:24.898547 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8ccfb478-e064-4773-b8d4-dba13853a2d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb62cc422387437dbef9bd444192cf06'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:24.993114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8ccfb478-e064-4773-b8d4-dba13853a2d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb62cc422387437dbef9bd444192cf06'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:24.993802 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8ccfb478-e064-4773-b8d4-dba13853a2d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb62cc422387437dbef9bd444192cf06'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:25.002297 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8ccfb478-e064-4773-b8d4-dba13853a2d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb62cc422387437dbef9bd444192cf06'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:25.020463 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1228/2459] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:43:24 2022] DELETE /identity/v3/users/eb62cc422387437dbef9bd444192cf06 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:43:25.297079 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718b7ae-55d2-41a3-ae35-55c4d6833c26 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:25.364796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718b7ae-55d2-41a3-ae35-55c4d6833c26 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:25.390542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:43:25.419472 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:25.420092 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:25.420363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:25.420629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] PATH_INFO: `/v3/projects/110490c08a2447a99f76d49efb277de6` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:25.421720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=110490c08a2447a99f76d49efb277de6)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:25.424372 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:25.558318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '110490c08a2447a99f76d49efb277de6', 'request_id': 'req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:25.564529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '110490c08a2447a99f76d49efb277de6', 'request_id': 'req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:25.647647 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '110490c08a2447a99f76d49efb277de6', 'request_id': 'req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:25.659310 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] The token cache is being invalidate because project 110490c08a2447a99f76d49efb277de6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:43:25.659599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:25.660233 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:25.661760 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1232/2460] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:43:25 2022] DELETE /identity/v3/projects/110490c08a2447a99f76d49efb277de6 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:43:25.950959 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ea51-e418-4078-aebc-ce1fc35bb5a2 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:25.951335 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ea51-e418-4078-aebc-ce1fc35bb5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:25.951731 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ea51-e418-4078-aebc-ce1fc35bb5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:25.974804 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-f991ea51-e418-4078-aebc-ce1fc35bb5a2 None None] MFA Rules not processed for user `717c74cb769148a5b465079add961892`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:43:25.976088 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f991ea51-e418-4078-aebc-ce1fc35bb5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:26.019419 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:43:26.039753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1229/2461] 10.222.0.20 () {58 vars in 967 bytes} [Wed Oct 19 02:43:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:43:26.052687 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:26.054086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:26.126925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b0143-50fb-4335-8c47-e0ec3e5de2e8 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:26.132154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9dba8d-450b-4984-b4c6-0609f22a476f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e476a32b458c4ea0bdc207d0ff4bfd47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:26.132750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dba8d-450b-4984-b4c6-0609f22a476f service swift] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:26.133023 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dba8d-450b-4984-b4c6-0609f22a476f service swift] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:26.133287 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9dba8d-450b-4984-b4c6-0609f22a476f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:26.141105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9dba8d-450b-4984-b4c6-0609f22a476f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:26.142723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9dba8d-450b-4984-b4c6-0609f22a476f service swift] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:26.150750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1233/2462] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Oct 19 02:43:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 19 02:43:36.118141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:43:36.147938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:43:45.909214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ea51-e418-4078-aebc-ce1fc35bb5a2 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:45.910658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-f991ea51-e418-4078-aebc-ce1fc35bb5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:45.973786 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ea51-e418-4078-aebc-ce1fc35bb5a2 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:45.978232 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:45.978845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:45.979123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:45.979390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] PATH_INFO: `/v3/users/bd40f521bebc4f85a1409e9f9a8936db` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:45.980517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd40f521bebc4f85a1409e9f9a8936db)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:45.982438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:46.152592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd40f521bebc4f85a1409e9f9a8936db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:46.153178 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd40f521bebc4f85a1409e9f9a8936db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:46.224083 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd40f521bebc4f85a1409e9f9a8936db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:46.224684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd40f521bebc4f85a1409e9f9a8936db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:46.232438 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd40f521bebc4f85a1409e9f9a8936db'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:46.249849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1230/2463] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:43:45 2022] DELETE /identity/v3/users/bd40f521bebc4f85a1409e9f9a8936db => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:43:46.254704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d5b64-c8db-4b2f-b7ad-b59bf768d138 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:46.255222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d5b64-c8db-4b2f-b7ad-b59bf768d138 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:46.255343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d5b64-c8db-4b2f-b7ad-b59bf768d138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:46.273802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-0d8d5b64-c8db-4b2f-b7ad-b59bf768d138 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:43:46.275020 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0d8d5b64-c8db-4b2f-b7ad-b59bf768d138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:46.332150 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1234/2464] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:43:46.342286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:46.343723 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:46.400887 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac1d99-af23-4a5b-8cef-fe593bb2e21a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:46.404644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f07b1-eb30-4c33-9ad4-42c9cf1179bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:46.405271 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f07b1-eb30-4c33-9ad4-42c9cf1179bd service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:46.405540 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f07b1-eb30-4c33-9ad4-42c9cf1179bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:46.405814 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481f07b1-eb30-4c33-9ad4-42c9cf1179bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:46.414856 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f07b1-eb30-4c33-9ad4-42c9cf1179bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:46.416275 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481f07b1-eb30-4c33-9ad4-42c9cf1179bd service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:46.426037 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1231/2465] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:43:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:43:46.834006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d5b64-c8db-4b2f-b7ad-b59bf768d138 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:43:46.835469 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0d8d5b64-c8db-4b2f-b7ad-b59bf768d138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:43:46.850252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d5b64-c8db-4b2f-b7ad-b59bf768d138 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:43:46.855495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:43:46.856226 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:43:46.856548 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:43:46.856938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] PATH_INFO: `/v3/projects/338befa62e384b6ab7ca5e66af885c06` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:43:46.858107 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=338befa62e384b6ab7ca5e66af885c06)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:43:46.864690 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:43:46.978896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '338befa62e384b6ab7ca5e66af885c06', 'request_id': 'req-b7c9d203-4a6c-4840-9981-dc42b9700327'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:46.984977 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '338befa62e384b6ab7ca5e66af885c06', 'request_id': 'req-b7c9d203-4a6c-4840-9981-dc42b9700327'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:47.060838 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '338befa62e384b6ab7ca5e66af885c06', 'request_id': 'req-b7c9d203-4a6c-4840-9981-dc42b9700327'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:47.071227 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] The token cache is being invalidate because project 338befa62e384b6ab7ca5e66af885c06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:43:47.071540 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:47.072155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:43:47.073646 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1235/2466] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:43:46 2022] DELETE /identity/v3/projects/338befa62e384b6ab7ca5e66af885c06 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:43:56.421898 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:43:57.068257 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:08.941755 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f07b1-eb30-4c33-9ad4-42c9cf1179bd service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:08.950361 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481f07b1-eb30-4c33-9ad4-42c9cf1179bd service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:08.998458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde75d39-eb4d-4d14-9fdc-86ed11489c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:08.999047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde75d39-eb4d-4d14-9fdc-86ed11489c00 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:08.999323 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde75d39-eb4d-4d14-9fdc-86ed11489c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:08.999594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde75d39-eb4d-4d14-9fdc-86ed11489c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:09.001803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-fde75d39-eb4d-4d14-9fdc-86ed11489c00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:09.063306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde75d39-eb4d-4d14-9fdc-86ed11489c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:09.064723 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde75d39-eb4d-4d14-9fdc-86ed11489c00 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:09.074096 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1232/2467] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:12.491652 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:12.493137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:12.507577 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9d203-4a6c-4840-9981-dc42b9700327 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:12.512235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac69b1a-6a6f-4ff1-a568-a9f4ca8bc154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:12.512872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac69b1a-6a6f-4ff1-a568-a9f4ca8bc154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:12.513132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac69b1a-6a6f-4ff1-a568-a9f4ca8bc154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:12.513406 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac69b1a-6a6f-4ff1-a568-a9f4ca8bc154 admin admin] PATH_INFO: `/v3/users/1d16e478c25946a8818641b7d5ea2e15` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:12.514568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac69b1a-6a6f-4ff1-a568-a9f4ca8bc154 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d16e478c25946a8818641b7d5ea2e15)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:12.516531 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac69b1a-6a6f-4ff1-a568-a9f4ca8bc154 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:12.683440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9ac69b1a-6a6f-4ff1-a568-a9f4ca8bc154 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d16e478c25946a8818641b7d5ea2e15'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:12.684036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9ac69b1a-6a6f-4ff1-a568-a9f4ca8bc154 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d16e478c25946a8818641b7d5ea2e15'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:12.690879 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9ac69b1a-6a6f-4ff1-a568-a9f4ca8bc154 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d16e478c25946a8818641b7d5ea2e15'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:12.755960 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9ac69b1a-6a6f-4ff1-a568-a9f4ca8bc154 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d16e478c25946a8818641b7d5ea2e15'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:12.756697 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-9ac69b1a-6a6f-4ff1-a568-a9f4ca8bc154 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d16e478c25946a8818641b7d5ea2e15'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:12.775700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1236/2468] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:44:12 2022] DELETE /identity/v3/users/1d16e478c25946a8818641b7d5ea2e15 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:13.031784 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde75d39-eb4d-4d14-9fdc-86ed11489c00 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:13.041735 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde75d39-eb4d-4d14-9fdc-86ed11489c00 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:13.099339 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:13.099970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:13.100250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:13.100523 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] PATH_INFO: `/v3/projects/0d7dfc2343ba4bd3a7f5f39a6765bb36` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:13.101656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d7dfc2343ba4bd3a7f5f39a6765bb36)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:13.103613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:13.250950 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d7dfc2343ba4bd3a7f5f39a6765bb36', 'request_id': 'req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:13.252785 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:13.257491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d7dfc2343ba4bd3a7f5f39a6765bb36', 'request_id': 'req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:13.335134 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d7dfc2343ba4bd3a7f5f39a6765bb36', 'request_id': 'req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:13.345966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] The token cache is being invalidate because project 0d7dfc2343ba4bd3a7f5f39a6765bb36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:44:13.346547 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:13.347357 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-3af60f77-6ded-4fbe-bd1c-f7ef843822a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:13.349123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1233/2469] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:44:13 2022] DELETE /identity/v3/projects/0d7dfc2343ba4bd3a7f5f39a6765bb36 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:13.354105 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a5e15-2038-4b4a-988c-cab7ecf78ce6 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:13.354390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a5e15-2038-4b4a-988c-cab7ecf78ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:13.354663 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a5e15-2038-4b4a-988c-cab7ecf78ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:13.375014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-9d0a5e15-2038-4b4a-988c-cab7ecf78ce6 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:13.376210 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-9d0a5e15-2038-4b4a-988c-cab7ecf78ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:13.432947 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1237/2470] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:13.448542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340e0ba-023f-4365-b5cb-484561bacdcc None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:13.448824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340e0ba-023f-4365-b5cb-484561bacdcc None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:13.449125 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0340e0ba-023f-4365-b5cb-484561bacdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:13.468472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-0340e0ba-023f-4365-b5cb-484561bacdcc None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:13.470229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0340e0ba-023f-4365-b5cb-484561bacdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:13.477866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1234/2471] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:13.482115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a5e15-2038-4b4a-988c-cab7ecf78ce6 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:13.492309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a5e15-2038-4b4a-988c-cab7ecf78ce6 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:13.496803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818028a-3d9c-4425-b9a4-10317c9ecf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:13.497392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818028a-3d9c-4425-b9a4-10317c9ecf31 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:13.497671 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818028a-3d9c-4425-b9a4-10317c9ecf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:13.497942 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8818028a-3d9c-4425-b9a4-10317c9ecf31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:13.498619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818028a-3d9c-4425-b9a4-10317c9ecf31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:44:13.498922 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818028a-3d9c-4425-b9a4-10317c9ecf31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:13.500521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8818028a-3d9c-4425-b9a4-10317c9ecf31 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:13.511950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1238/2472] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:44:13.516161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340e0ba-023f-4365-b5cb-484561bacdcc None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:13.525260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0340e0ba-023f-4365-b5cb-484561bacdcc None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:13.529797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c57713-7b5f-4f33-aabc-e87375387e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:13.530412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57713-7b5f-4f33-aabc-e87375387e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:13.530686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57713-7b5f-4f33-aabc-e87375387e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:13.530952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c57713-7b5f-4f33-aabc-e87375387e03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:13.532187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c57713-7b5f-4f33-aabc-e87375387e03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:13.534266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c57713-7b5f-4f33-aabc-e87375387e03 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:13.645827 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1235/2473] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:44:13 2022] POST /identity/v3/projects => generated 357 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:13.649740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818028a-3d9c-4425-b9a4-10317c9ecf31 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:13.660225 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8818028a-3d9c-4425-b9a4-10317c9ecf31 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:13.715566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74432a04-8b73-4e1e-a427-34c54259378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:13.716161 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74432a04-8b73-4e1e-a427-34c54259378c admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:13.716444 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74432a04-8b73-4e1e-a427-34c54259378c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:13.716716 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74432a04-8b73-4e1e-a427-34c54259378c admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:13.717341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74432a04-8b73-4e1e-a427-34c54259378c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:13.719896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74432a04-8b73-4e1e-a427-34c54259378c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:13.727726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:13.789488 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1239/2474] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:44:13 2022] POST /identity/v3/users => generated 279 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:13.793625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c57713-7b5f-4f33-aabc-e87375387e03 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:13.804553 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c57713-7b5f-4f33-aabc-e87375387e03 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:13.809158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b5827-3c99-4b2b-8bbe-47dbd2369fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:13.809753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b5827-3c99-4b2b-8bbe-47dbd2369fbd admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:13.810027 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b5827-3c99-4b2b-8bbe-47dbd2369fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:13.810332 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b5827-3c99-4b2b-8bbe-47dbd2369fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:13.810970 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b5827-3c99-4b2b-8bbe-47dbd2369fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:13.813089 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b5827-3c99-4b2b-8bbe-47dbd2369fbd admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:13.823002 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1236/2475] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:13.827025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74432a04-8b73-4e1e-a427-34c54259378c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:13.837707 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74432a04-8b73-4e1e-a427-34c54259378c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:13.842506 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450101f1-d336-4cf5-b76f-35abd2c5feb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:13.843087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450101f1-d336-4cf5-b76f-35abd2c5feb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:13.843361 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450101f1-d336-4cf5-b76f-35abd2c5feb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:13.843627 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450101f1-d336-4cf5-b76f-35abd2c5feb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:13.844384 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450101f1-d336-4cf5-b76f-35abd2c5feb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:13.846118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450101f1-d336-4cf5-b76f-35abd2c5feb5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:13.856542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1240/2476] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:13.861106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b5827-3c99-4b2b-8bbe-47dbd2369fbd admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:13.870934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b5827-3c99-4b2b-8bbe-47dbd2369fbd admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:13.876041 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1743ab-5e09-4d63-9888-048865b1ba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:13.876688 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1743ab-5e09-4d63-9888-048865b1ba04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:13.876965 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1743ab-5e09-4d63-9888-048865b1ba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:13.877239 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1743ab-5e09-4d63-9888-048865b1ba04 admin admin] PATH_INFO: `/v3/projects/6b30dc8f76584a47b5030e60b14281af/users/cd670f4bbd064ca391f23fdba94a3c27/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:13.891798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1743ab-5e09-4d63-9888-048865b1ba04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b30dc8f76584a47b5030e60b14281af, user_id=cd670f4bbd064ca391f23fdba94a3c27, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:13.894467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1743ab-5e09-4d63-9888-048865b1ba04 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:13.985142 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1237/2477] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:13 2022] PUT /identity/v3/projects/6b30dc8f76584a47b5030e60b14281af/users/cd670f4bbd064ca391f23fdba94a3c27/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:13.995555 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b88ac-af32-47d7-b4ce-9b803cfa2daa None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:13.995847 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b88ac-af32-47d7-b4ce-9b803cfa2daa None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:13.996123 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b88ac-af32-47d7-b4ce-9b803cfa2daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:14.023975 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-284b88ac-af32-47d7-b4ce-9b803cfa2daa None None] MFA Rules not processed for user `cd670f4bbd064ca391f23fdba94a3c27`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:14.026535 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-284b88ac-af32-47d7-b4ce-9b803cfa2daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:14.087807 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1241/2478] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:14.092261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1743ab-5e09-4d63-9888-048865b1ba04 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:14.155239 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1743ab-5e09-4d63-9888-048865b1ba04 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:14.160514 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafcf9e-dea4-4d65-9ed7-afc3b9f91181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:14.161112 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafcf9e-dea4-4d65-9ed7-afc3b9f91181 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:14.161393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafcf9e-dea4-4d65-9ed7-afc3b9f91181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:14.161667 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafcf9e-dea4-4d65-9ed7-afc3b9f91181 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:14.162325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafcf9e-dea4-4d65-9ed7-afc3b9f91181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:14.164223 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafcf9e-dea4-4d65-9ed7-afc3b9f91181 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:14.250782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1238/2479] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:44:14 2022] POST /identity/v3/projects => generated 357 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:14.255274 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b88ac-af32-47d7-b4ce-9b803cfa2daa None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:14.266137 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b88ac-af32-47d7-b4ce-9b803cfa2daa None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:14.321520 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e937108-9e3a-401a-83ac-5e93300597a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:14.322151 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e937108-9e3a-401a-83ac-5e93300597a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:14.322426 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e937108-9e3a-401a-83ac-5e93300597a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:14.322693 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e937108-9e3a-401a-83ac-5e93300597a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:14.323315 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e937108-9e3a-401a-83ac-5e93300597a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:14.326163 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e937108-9e3a-401a-83ac-5e93300597a4 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:14.444490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1242/2480] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:44:14 2022] POST /identity/v3/users => generated 285 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:14.448126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafcf9e-dea4-4d65-9ed7-afc3b9f91181 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:14.458879 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafcf9e-dea4-4d65-9ed7-afc3b9f91181 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:14.463525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9a5f3-360d-480c-8b43-34d221efe05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:14.464123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a5f3-360d-480c-8b43-34d221efe05f admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:14.464859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a5f3-360d-480c-8b43-34d221efe05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:14.464859 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a5f3-360d-480c-8b43-34d221efe05f admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:14.465324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9a5f3-360d-480c-8b43-34d221efe05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:14.466958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9a5f3-360d-480c-8b43-34d221efe05f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:14.478372 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1239/2481] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:14.482493 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e937108-9e3a-401a-83ac-5e93300597a4 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:14.492497 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e937108-9e3a-401a-83ac-5e93300597a4 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:14.497125 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42758d-5d48-4400-b9d5-983292628a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:14.497776 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42758d-5d48-4400-b9d5-983292628a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:14.498070 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42758d-5d48-4400-b9d5-983292628a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:14.498352 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42758d-5d48-4400-b9d5-983292628a7d admin admin] PATH_INFO: `/v3/projects/9b21aeef27e544bdaba931bb26efd229/users/22ef94d0a9bf441fbb7277b071354d55/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:14.513053 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42758d-5d48-4400-b9d5-983292628a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b21aeef27e544bdaba931bb26efd229, user_id=22ef94d0a9bf441fbb7277b071354d55, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:14.516218 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42758d-5d48-4400-b9d5-983292628a7d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:14.601583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1243/2482] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:14 2022] PUT /identity/v3/projects/9b21aeef27e544bdaba931bb26efd229/users/22ef94d0a9bf441fbb7277b071354d55/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:14.605512 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9a5f3-360d-480c-8b43-34d221efe05f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:14.615496 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9a5f3-360d-480c-8b43-34d221efe05f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:14.661500 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:14.670988 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a3c41-396f-488f-a2c5-ed940be64fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:14.671622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a3c41-396f-488f-a2c5-ed940be64fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:14.671958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a3c41-396f-488f-a2c5-ed940be64fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:14.672233 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a3c41-396f-488f-a2c5-ed940be64fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:14.672861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a3c41-396f-488f-a2c5-ed940be64fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:14.674483 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a3c41-396f-488f-a2c5-ed940be64fa4 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:14.684161 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1240/2483] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:14.688308 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42758d-5d48-4400-b9d5-983292628a7d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:14.700244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42758d-5d48-4400-b9d5-983292628a7d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:14.703579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840c9fb-47f7-469a-bcb9-b2f8798138aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:14.704217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840c9fb-47f7-469a-bcb9-b2f8798138aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:14.704495 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840c9fb-47f7-469a-bcb9-b2f8798138aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:14.704773 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840c9fb-47f7-469a-bcb9-b2f8798138aa admin admin] PATH_INFO: `/v3/projects/9b21aeef27e544bdaba931bb26efd229/users/22ef94d0a9bf441fbb7277b071354d55/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:14.706740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840c9fb-47f7-469a-bcb9-b2f8798138aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b21aeef27e544bdaba931bb26efd229, user_id=22ef94d0a9bf441fbb7277b071354d55, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:14.709356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840c9fb-47f7-469a-bcb9-b2f8798138aa admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:14.720161 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1244/2484] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:14 2022] PUT /identity/v3/projects/9b21aeef27e544bdaba931bb26efd229/users/22ef94d0a9bf441fbb7277b071354d55/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:14.730337 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266fae4-f3ea-4411-8d30-1c9e2339fe2b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:14.730622 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266fae4-f3ea-4411-8d30-1c9e2339fe2b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:14.730896 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266fae4-f3ea-4411-8d30-1c9e2339fe2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:14.758686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e266fae4-f3ea-4411-8d30-1c9e2339fe2b None None] MFA Rules not processed for user `22ef94d0a9bf441fbb7277b071354d55`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:14.759895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e266fae4-f3ea-4411-8d30-1c9e2339fe2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:14.826226 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1241/2485] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:14.837755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840c9fb-47f7-469a-bcb9-b2f8798138aa admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:14.839454 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a840c9fb-47f7-469a-bcb9-b2f8798138aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:14.861179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:14.896496 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840c9fb-47f7-469a-bcb9-b2f8798138aa admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:14.900124 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457bc7a-6285-40c2-8c01-521f1aeb0a06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:14.900722 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457bc7a-6285-40c2-8c01-521f1aeb0a06 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:14.900994 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457bc7a-6285-40c2-8c01-521f1aeb0a06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:14.901267 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457bc7a-6285-40c2-8c01-521f1aeb0a06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:14.910362 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457bc7a-6285-40c2-8c01-521f1aeb0a06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:14.911868 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457bc7a-6285-40c2-8c01-521f1aeb0a06 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:14.921287 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1245/2486] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:15.135513 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266fae4-f3ea-4411-8d30-1c9e2339fe2b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:15.145738 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266fae4-f3ea-4411-8d30-1c9e2339fe2b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:15.149229 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a685aa-a12d-43e8-bd63-933ab694d4bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:15.149822 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a685aa-a12d-43e8-bd63-933ab694d4bb service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:15.150201 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a685aa-a12d-43e8-bd63-933ab694d4bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:15.150476 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a685aa-a12d-43e8-bd63-933ab694d4bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:15.207506 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a685aa-a12d-43e8-bd63-933ab694d4bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:15.209007 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a685aa-a12d-43e8-bd63-933ab694d4bb service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:15.217705 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1242/2487] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:24.952608 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=17 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:25.213644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=68 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:34.153021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457bc7a-6285-40c2-8c01-521f1aeb0a06 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:34.213851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457bc7a-6285-40c2-8c01-521f1aeb0a06 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:34.218305 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:34.219007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:34.219566 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:34.219840 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] PATH_INFO: `/v3/users/cd670f4bbd064ca391f23fdba94a3c27` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:34.220941 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd670f4bbd064ca391f23fdba94a3c27)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:34.222848 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:34.377770 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd670f4bbd064ca391f23fdba94a3c27'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:34.378409 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd670f4bbd064ca391f23fdba94a3c27'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:34.385802 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd670f4bbd064ca391f23fdba94a3c27'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:34.443818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd670f4bbd064ca391f23fdba94a3c27'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:34.444437 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd670f4bbd064ca391f23fdba94a3c27'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:34.461992 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1246/2488] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:44:34 2022] DELETE /identity/v3/users/cd670f4bbd064ca391f23fdba94a3c27 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:34.466509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab182e-5408-4606-aac2-bc50c83bce47 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:34.466986 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab182e-5408-4606-aac2-bc50c83bce47 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:34.467113 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab182e-5408-4606-aac2-bc50c83bce47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:34.486293 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-c2ab182e-5408-4606-aac2-bc50c83bce47 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:34.487492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c2ab182e-5408-4606-aac2-bc50c83bce47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:34.545141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1243/2489] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:34.555539 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:34.556934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:34.613116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d75425-d791-4029-b8e9-5ffe48221f7c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:34.616805 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58d260-5b17-4663-9635-74a2a87029cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:34.617403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58d260-5b17-4663-9635-74a2a87029cd service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:34.617683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58d260-5b17-4663-9635-74a2a87029cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:34.617957 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58d260-5b17-4663-9635-74a2a87029cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:34.626845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58d260-5b17-4663-9635-74a2a87029cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:34.628299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58d260-5b17-4663-9635-74a2a87029cd service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:34.637898 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1247/2490] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:35.034354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab182e-5408-4606-aac2-bc50c83bce47 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:35.035861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c2ab182e-5408-4606-aac2-bc50c83bce47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:35.047741 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab182e-5408-4606-aac2-bc50c83bce47 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:35.052459 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:35.053094 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:35.053365 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:35.053632 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] PATH_INFO: `/v3/projects/6b30dc8f76584a47b5030e60b14281af` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:35.054828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b30dc8f76584a47b5030e60b14281af)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:35.059325 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:35.158882 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b30dc8f76584a47b5030e60b14281af', 'request_id': 'req-344bbd68-97be-4563-b8d3-bae7caf8abb7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:35.165029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b30dc8f76584a47b5030e60b14281af', 'request_id': 'req-344bbd68-97be-4563-b8d3-bae7caf8abb7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:35.226363 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b30dc8f76584a47b5030e60b14281af', 'request_id': 'req-344bbd68-97be-4563-b8d3-bae7caf8abb7'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:35.236736 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] The token cache is being invalidate because project 6b30dc8f76584a47b5030e60b14281af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:44:35.237019 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:35.237660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:35.239166 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1244/2491] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:44:35 2022] DELETE /identity/v3/projects/6b30dc8f76584a47b5030e60b14281af => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:35.243174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58d260-5b17-4663-9635-74a2a87029cd service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:35.253168 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58d260-5b17-4663-9635-74a2a87029cd service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:35.272393 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:35.309629 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840633c-e1ca-49c4-b065-a8fcaf6591b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:35.310444 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840633c-e1ca-49c4-b065-a8fcaf6591b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:35.310719 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840633c-e1ca-49c4-b065-a8fcaf6591b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:35.313328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840633c-e1ca-49c4-b065-a8fcaf6591b1 admin admin] PATH_INFO: `/v3/users/22ef94d0a9bf441fbb7277b071354d55` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:35.313328 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840633c-e1ca-49c4-b065-a8fcaf6591b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22ef94d0a9bf441fbb7277b071354d55)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:35.314131 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840633c-e1ca-49c4-b065-a8fcaf6591b1 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:35.522502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c840633c-e1ca-49c4-b065-a8fcaf6591b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22ef94d0a9bf441fbb7277b071354d55'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:35.526110 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c840633c-e1ca-49c4-b065-a8fcaf6591b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22ef94d0a9bf441fbb7277b071354d55'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:35.530071 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c840633c-e1ca-49c4-b065-a8fcaf6591b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22ef94d0a9bf441fbb7277b071354d55'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:35.618422 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c840633c-e1ca-49c4-b065-a8fcaf6591b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22ef94d0a9bf441fbb7277b071354d55'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:35.619066 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c840633c-e1ca-49c4-b065-a8fcaf6591b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22ef94d0a9bf441fbb7277b071354d55'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:35.640511 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1248/2492] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:44:35 2022] DELETE /identity/v3/users/22ef94d0a9bf441fbb7277b071354d55 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:36.025492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:36.026968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:36.040343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bbd68-97be-4563-b8d3-bae7caf8abb7 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:36.101865 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:36.102527 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:36.102803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:36.103071 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] PATH_INFO: `/v3/projects/9b21aeef27e544bdaba931bb26efd229` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:36.104186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b21aeef27e544bdaba931bb26efd229)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:36.106097 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:36.213945 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:36.230163 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b21aeef27e544bdaba931bb26efd229', 'request_id': 'req-8c7f6adc-78d7-4c89-acce-4337feb14ce9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:36.236184 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b21aeef27e544bdaba931bb26efd229', 'request_id': 'req-8c7f6adc-78d7-4c89-acce-4337feb14ce9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:36.315704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b21aeef27e544bdaba931bb26efd229', 'request_id': 'req-8c7f6adc-78d7-4c89-acce-4337feb14ce9'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:36.326825 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] The token cache is being invalidate because project 9b21aeef27e544bdaba931bb26efd229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:44:36.327106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:36.327722 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-8c7f6adc-78d7-4c89-acce-4337feb14ce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:36.329185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1245/2493] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:44:36 2022] DELETE /identity/v3/projects/9b21aeef27e544bdaba931bb26efd229 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:36.334660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aaa29a-f5ca-4c67-90d3-971cc83f72d4 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:36.335154 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aaa29a-f5ca-4c67-90d3-971cc83f72d4 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:36.335265 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aaa29a-f5ca-4c67-90d3-971cc83f72d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:36.355036 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-28aaa29a-f5ca-4c67-90d3-971cc83f72d4 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:36.356226 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-28aaa29a-f5ca-4c67-90d3-971cc83f72d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:36.415919 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1249/2494] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:36.423723 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8781d8-c00e-4bdf-932e-57122561da61 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:36.424001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8781d8-c00e-4bdf-932e-57122561da61 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:36.424270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8781d8-c00e-4bdf-932e-57122561da61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:36.443972 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-4c8781d8-c00e-4bdf-932e-57122561da61 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:36.445141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4c8781d8-c00e-4bdf-932e-57122561da61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:36.453165 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1246/2495] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:36.457177 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aaa29a-f5ca-4c67-90d3-971cc83f72d4 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:36.467265 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aaa29a-f5ca-4c67-90d3-971cc83f72d4 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:36.471762 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4c04e-8a76-4c5a-a389-7329a2871e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:36.472356 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4c04e-8a76-4c5a-a389-7329a2871e5b admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:36.472633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4c04e-8a76-4c5a-a389-7329a2871e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:36.472903 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4c04e-8a76-4c5a-a389-7329a2871e5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:36.473570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4c04e-8a76-4c5a-a389-7329a2871e5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:44:36.473872 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4c04e-8a76-4c5a-a389-7329a2871e5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:36.475487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4c04e-8a76-4c5a-a389-7329a2871e5b admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:36.486656 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1250/2496] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:44:36.490878 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8781d8-c00e-4bdf-932e-57122561da61 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:36.499868 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8781d8-c00e-4bdf-932e-57122561da61 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:36.505455 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848bcaf-1997-460c-aa8b-43943a870675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:36.506035 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848bcaf-1997-460c-aa8b-43943a870675 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:36.506328 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848bcaf-1997-460c-aa8b-43943a870675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:36.506594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848bcaf-1997-460c-aa8b-43943a870675 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:36.507209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848bcaf-1997-460c-aa8b-43943a870675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:36.509074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848bcaf-1997-460c-aa8b-43943a870675 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:36.625466 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1247/2497] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:44:36 2022] POST /identity/v3/projects => generated 361 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:36.630887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4c04e-8a76-4c5a-a389-7329a2871e5b admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:36.640968 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4c04e-8a76-4c5a-a389-7329a2871e5b admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:36.667091 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:36.696332 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb83a2-7600-439b-87a7-4cff0a82598c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:36.696934 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb83a2-7600-439b-87a7-4cff0a82598c admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:36.697217 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb83a2-7600-439b-87a7-4cff0a82598c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:36.697492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb83a2-7600-439b-87a7-4cff0a82598c admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:36.698164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb83a2-7600-439b-87a7-4cff0a82598c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:36.700073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb83a2-7600-439b-87a7-4cff0a82598c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:36.779180 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1251/2498] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:44:36 2022] POST /identity/v3/users => generated 281 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:36.783687 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848bcaf-1997-460c-aa8b-43943a870675 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:36.794370 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848bcaf-1997-460c-aa8b-43943a870675 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:36.798995 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400bf48-731e-4676-be91-36d66c31c89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:36.799588 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400bf48-731e-4676-be91-36d66c31c89b admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:36.799862 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400bf48-731e-4676-be91-36d66c31c89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:36.800128 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400bf48-731e-4676-be91-36d66c31c89b admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:36.800754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400bf48-731e-4676-be91-36d66c31c89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:36.802412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400bf48-731e-4676-be91-36d66c31c89b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:36.812139 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1248/2499] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:36.816252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb83a2-7600-439b-87a7-4cff0a82598c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:36.827116 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb83a2-7600-439b-87a7-4cff0a82598c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:36.831826 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a32ca-f0eb-44be-af05-b6f3807ff24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:36.832424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a32ca-f0eb-44be-af05-b6f3807ff24a admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:36.833341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a32ca-f0eb-44be-af05-b6f3807ff24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:36.833611 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a32ca-f0eb-44be-af05-b6f3807ff24a admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:36.834282 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a32ca-f0eb-44be-af05-b6f3807ff24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:36.835849 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a32ca-f0eb-44be-af05-b6f3807ff24a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:36.846666 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1252/2500] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:36.850710 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400bf48-731e-4676-be91-36d66c31c89b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:36.860501 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400bf48-731e-4676-be91-36d66c31c89b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:36.865490 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a93e2-a9a5-4fa7-8904-eca11cc3715c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:36.866156 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a93e2-a9a5-4fa7-8904-eca11cc3715c admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:36.866434 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a93e2-a9a5-4fa7-8904-eca11cc3715c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:36.866706 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a93e2-a9a5-4fa7-8904-eca11cc3715c admin admin] PATH_INFO: `/v3/projects/3de5cfa294de435ea7df3bf9c51b5e7d/users/cf9d2d11c9674a62ac191fc914619294/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:36.881828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a93e2-a9a5-4fa7-8904-eca11cc3715c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3de5cfa294de435ea7df3bf9c51b5e7d, user_id=cf9d2d11c9674a62ac191fc914619294, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:36.884557 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a93e2-a9a5-4fa7-8904-eca11cc3715c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:36.937487 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1249/2501] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:36 2022] PUT /identity/v3/projects/3de5cfa294de435ea7df3bf9c51b5e7d/users/cf9d2d11c9674a62ac191fc914619294/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:36.948302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d984e7-c656-4610-a4fe-bbee267e06ed None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:36.948580 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d984e7-c656-4610-a4fe-bbee267e06ed None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:36.948853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d984e7-c656-4610-a4fe-bbee267e06ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:36.976010 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-66d984e7-c656-4610-a4fe-bbee267e06ed None None] MFA Rules not processed for user `cf9d2d11c9674a62ac191fc914619294`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:36.977219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-66d984e7-c656-4610-a4fe-bbee267e06ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:37.040851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1253/2502] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:37.045341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a93e2-a9a5-4fa7-8904-eca11cc3715c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:37.107858 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a93e2-a9a5-4fa7-8904-eca11cc3715c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:37.112330 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de5131-6fbc-491b-9b80-e483d7dcd24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:37.112877 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de5131-6fbc-491b-9b80-e483d7dcd24a admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:37.113157 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de5131-6fbc-491b-9b80-e483d7dcd24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:37.113433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09de5131-6fbc-491b-9b80-e483d7dcd24a admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:37.114634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de5131-6fbc-491b-9b80-e483d7dcd24a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:37.116548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09de5131-6fbc-491b-9b80-e483d7dcd24a admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:37.201603 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1250/2503] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:44:37 2022] POST /identity/v3/projects => generated 363 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:37.205779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d984e7-c656-4610-a4fe-bbee267e06ed None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:37.216354 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d984e7-c656-4610-a4fe-bbee267e06ed None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:37.270859 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2aee-8b66-40b4-b403-bd557a254155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:37.271452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2aee-8b66-40b4-b403-bd557a254155 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:37.271729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2aee-8b66-40b4-b403-bd557a254155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:37.271998 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c2aee-8b66-40b4-b403-bd557a254155 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:37.272624 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2aee-8b66-40b4-b403-bd557a254155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:37.274522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c2aee-8b66-40b4-b403-bd557a254155 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:37.346553 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1254/2504] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:44:37 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:37.351141 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de5131-6fbc-491b-9b80-e483d7dcd24a admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:37.361803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09de5131-6fbc-491b-9b80-e483d7dcd24a admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:37.367177 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da5a16-9119-4967-aa67-ecebeac00ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:37.367720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5a16-9119-4967-aa67-ecebeac00ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:37.368000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5a16-9119-4967-aa67-ecebeac00ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:37.368272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5a16-9119-4967-aa67-ecebeac00ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:37.368936 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da5a16-9119-4967-aa67-ecebeac00ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:37.370601 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da5a16-9119-4967-aa67-ecebeac00ad6 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:37.381203 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1251/2505] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:44:37.385449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2aee-8b66-40b4-b403-bd557a254155 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:37.395321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c2aee-8b66-40b4-b403-bd557a254155 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:37.399828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30cd5-1d01-4da0-bfcb-d452cbbd0f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:37.400464 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30cd5-1d01-4da0-bfcb-d452cbbd0f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:37.400737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30cd5-1d01-4da0-bfcb-d452cbbd0f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:37.401006 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a30cd5-1d01-4da0-bfcb-d452cbbd0f3c admin admin] PATH_INFO: `/v3/projects/32e92238869349aa9e43672980389588/users/eb36800b5a7541969707d7d897d80c29/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:37.416272 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a30cd5-1d01-4da0-bfcb-d452cbbd0f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e92238869349aa9e43672980389588, user_id=eb36800b5a7541969707d7d897d80c29, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:37.419019 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a30cd5-1d01-4da0-bfcb-d452cbbd0f3c admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:37.472866 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1255/2506] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:37 2022] PUT /identity/v3/projects/32e92238869349aa9e43672980389588/users/eb36800b5a7541969707d7d897d80c29/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:37.477382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da5a16-9119-4967-aa67-ecebeac00ad6 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:37.487517 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da5a16-9119-4967-aa67-ecebeac00ad6 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:37.507797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:37.543429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94173af2-dc44-418f-9fca-47e22f184004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:37.544029 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94173af2-dc44-418f-9fca-47e22f184004 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:37.544310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94173af2-dc44-418f-9fca-47e22f184004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:37.544583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94173af2-dc44-418f-9fca-47e22f184004 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:37.545215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94173af2-dc44-418f-9fca-47e22f184004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:37.546847 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94173af2-dc44-418f-9fca-47e22f184004 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:37.556897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1252/2507] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:37.561027 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30cd5-1d01-4da0-bfcb-d452cbbd0f3c admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:37.566363 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:37.571738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a30cd5-1d01-4da0-bfcb-d452cbbd0f3c admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:37.576424 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64d116-ce0e-488a-9833-788203962394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:37.577040 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64d116-ce0e-488a-9833-788203962394 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:37.577318 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64d116-ce0e-488a-9833-788203962394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:37.577591 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64d116-ce0e-488a-9833-788203962394 admin admin] PATH_INFO: `/v3/projects/32e92238869349aa9e43672980389588/users/eb36800b5a7541969707d7d897d80c29/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:37.579542 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64d116-ce0e-488a-9833-788203962394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e92238869349aa9e43672980389588, user_id=eb36800b5a7541969707d7d897d80c29, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:37.582200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f64d116-ce0e-488a-9833-788203962394 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:37.595930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1256/2508] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:37 2022] PUT /identity/v3/projects/32e92238869349aa9e43672980389588/users/eb36800b5a7541969707d7d897d80c29/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:37.667720 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91960160-79f6-496d-a199-83e5ea968029 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:37.667966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91960160-79f6-496d-a199-83e5ea968029 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:37.668205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91960160-79f6-496d-a199-83e5ea968029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:37.696542 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-91960160-79f6-496d-a199-83e5ea968029 None None] MFA Rules not processed for user `eb36800b5a7541969707d7d897d80c29`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:37.697739 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-91960160-79f6-496d-a199-83e5ea968029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:37.763037 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1253/2509] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:37.774906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64d116-ce0e-488a-9833-788203962394 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:37.776673 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-2f64d116-ce0e-488a-9833-788203962394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:37.834288 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64d116-ce0e-488a-9833-788203962394 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:37.837969 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b8179-64c0-4a09-9d97-24501ba06686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:37.838600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b8179-64c0-4a09-9d97-24501ba06686 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:37.838881 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b8179-64c0-4a09-9d97-24501ba06686 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:37.839158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b8179-64c0-4a09-9d97-24501ba06686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:37.848115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b8179-64c0-4a09-9d97-24501ba06686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:37.850139 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b8179-64c0-4a09-9d97-24501ba06686 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:37.859490 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1257/2510] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:38.094607 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91960160-79f6-496d-a199-83e5ea968029 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:38.105266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91960160-79f6-496d-a199-83e5ea968029 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:38.108924 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43dadfe-36da-48b4-84bf-a54fcee806f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:38.109524 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dadfe-36da-48b4-84bf-a54fcee806f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:38.109811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dadfe-36da-48b4-84bf-a54fcee806f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:38.110158 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43dadfe-36da-48b4-84bf-a54fcee806f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:38.164973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dadfe-36da-48b4-84bf-a54fcee806f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:38.166439 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43dadfe-36da-48b4-84bf-a54fcee806f3 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:38.175047 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1254/2511] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:41.578734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b8179-64c0-4a09-9d97-24501ba06686 service cinder] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:41.643344 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b8179-64c0-4a09-9d97-24501ba06686 service cinder] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:41.648577 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:41.649244 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:41.649522 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:41.649794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] PATH_INFO: `/v3/users/cf9d2d11c9674a62ac191fc914619294` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:41.650927 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf9d2d11c9674a62ac191fc914619294)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:41.652854 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:41.829169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf9d2d11c9674a62ac191fc914619294'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:41.829828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf9d2d11c9674a62ac191fc914619294'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:41.837173 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf9d2d11c9674a62ac191fc914619294'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:41.926419 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf9d2d11c9674a62ac191fc914619294'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:41.926964 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf9d2d11c9674a62ac191fc914619294'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:41.944699 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1258/2512] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:44:41 2022] DELETE /identity/v3/users/cf9d2d11c9674a62ac191fc914619294 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:41.949218 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817899e6-a084-4b15-99e4-21e7f6a5326b None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:41.949726 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817899e6-a084-4b15-99e4-21e7f6a5326b None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:41.949844 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817899e6-a084-4b15-99e4-21e7f6a5326b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:41.969834 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-817899e6-a084-4b15-99e4-21e7f6a5326b None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:41.971288 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-817899e6-a084-4b15-99e4-21e7f6a5326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:42.009681 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=100 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:42.027613 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1255/2513] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:42.037313 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:42.038779 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:42.052073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:42.095436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac9b0b-0a6c-4937-a9e8-7874bf01e73d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:42.099067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96512621-333e-49e9-a6a1-ccb9fe53902c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:42.099672 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96512621-333e-49e9-a6a1-ccb9fe53902c service neutron] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:42.099951 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96512621-333e-49e9-a6a1-ccb9fe53902c service neutron] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:42.100221 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96512621-333e-49e9-a6a1-ccb9fe53902c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:42.109732 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96512621-333e-49e9-a6a1-ccb9fe53902c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:42.111211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96512621-333e-49e9-a6a1-ccb9fe53902c service neutron] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:42.121292 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1259/2514] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:42.543234 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817899e6-a084-4b15-99e4-21e7f6a5326b None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:42.544651 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-817899e6-a084-4b15-99e4-21e7f6a5326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:42.559283 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817899e6-a084-4b15-99e4-21e7f6a5326b None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:42.563934 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:42.565257 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:42.565894 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:42.566208 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] PATH_INFO: `/v3/projects/3de5cfa294de435ea7df3bf9c51b5e7d` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:42.567414 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=3de5cfa294de435ea7df3bf9c51b5e7d)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:42.569306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:42.701912 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3de5cfa294de435ea7df3bf9c51b5e7d', 'request_id': 'req-4893b025-536c-4387-b407-9726ad1c1cde'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:42.708495 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3de5cfa294de435ea7df3bf9c51b5e7d', 'request_id': 'req-4893b025-536c-4387-b407-9726ad1c1cde'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:42.832719 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3de5cfa294de435ea7df3bf9c51b5e7d', 'request_id': 'req-4893b025-536c-4387-b407-9726ad1c1cde'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:42.843987 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] The token cache is being invalidate because project 3de5cfa294de435ea7df3bf9c51b5e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:44:42.844277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:42.844906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:42.846427 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1256/2515] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:44:42 2022] DELETE /identity/v3/projects/3de5cfa294de435ea7df3bf9c51b5e7d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:42.851156 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96512621-333e-49e9-a6a1-ccb9fe53902c service neutron] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:42.860782 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96512621-333e-49e9-a6a1-ccb9fe53902c service neutron] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:42.915630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581918ee-398c-485f-aa44-c44cd1d2934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:42.916246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581918ee-398c-485f-aa44-c44cd1d2934f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:42.916525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581918ee-398c-485f-aa44-c44cd1d2934f admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:42.916797 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581918ee-398c-485f-aa44-c44cd1d2934f admin admin] PATH_INFO: `/v3/users/eb36800b5a7541969707d7d897d80c29` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:42.917907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581918ee-398c-485f-aa44-c44cd1d2934f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb36800b5a7541969707d7d897d80c29)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:42.919827 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581918ee-398c-485f-aa44-c44cd1d2934f admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:43.129174 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-581918ee-398c-485f-aa44-c44cd1d2934f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb36800b5a7541969707d7d897d80c29'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:43.130098 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-581918ee-398c-485f-aa44-c44cd1d2934f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb36800b5a7541969707d7d897d80c29'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:43.137734 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-581918ee-398c-485f-aa44-c44cd1d2934f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb36800b5a7541969707d7d897d80c29'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:43.220691 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-581918ee-398c-485f-aa44-c44cd1d2934f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb36800b5a7541969707d7d897d80c29'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:43.221302 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-581918ee-398c-485f-aa44-c44cd1d2934f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb36800b5a7541969707d7d897d80c29'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:43.238655 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1260/2516] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:44:42 2022] DELETE /identity/v3/users/eb36800b5a7541969707d7d897d80c29 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:43.720210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:43.721729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:43.734863 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893b025-536c-4387-b407-9726ad1c1cde admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:43.791341 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:43.791968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:43.792269 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:43.792520 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] PATH_INFO: `/v3/projects/32e92238869349aa9e43672980389588` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:43.793634 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32e92238869349aa9e43672980389588)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:43.795586 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:43.906255 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32e92238869349aa9e43672980389588', 'request_id': 'req-e3d968e2-7944-4081-87b1-99263ec36401'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:43.912541 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32e92238869349aa9e43672980389588', 'request_id': 'req-e3d968e2-7944-4081-87b1-99263ec36401'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:44.004445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32e92238869349aa9e43672980389588', 'request_id': 'req-e3d968e2-7944-4081-87b1-99263ec36401'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:44.011415 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:44.016418 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] The token cache is being invalidate because project 32e92238869349aa9e43672980389588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81615) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:44:44.016701 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:44.017343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-e3d968e2-7944-4081-87b1-99263ec36401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:44.018866 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1257/2517] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:44:43 2022] DELETE /identity/v3/projects/32e92238869349aa9e43672980389588 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:44.023793 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c5fe-c749-4285-bdce-32665f79d1be None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:44.024294 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c5fe-c749-4285-bdce-32665f79d1be None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:44.024403 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313c5fe-c749-4285-bdce-32665f79d1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:44.044349 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-8313c5fe-c749-4285-bdce-32665f79d1be None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:44.045523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-8313c5fe-c749-4285-bdce-32665f79d1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:44.073950 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:44.102925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1261/2518] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:44:44.110492 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067d40f-c09e-46a7-9ad9-9693e209458e None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:44.110801 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067d40f-c09e-46a7-9ad9-9693e209458e None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:44.112704 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067d40f-c09e-46a7-9ad9-9693e209458e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:44.131693 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-6067d40f-c09e-46a7-9ad9-9693e209458e None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:44.132884 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6067d40f-c09e-46a7-9ad9-9693e209458e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:44.140676 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1258/2519] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:44.144808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313c5fe-c749-4285-bdce-32665f79d1be None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:44.154383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313c5fe-c749-4285-bdce-32665f79d1be None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:44.158932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffe6b6-bd6d-497a-837e-d747504b4fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:44.159585 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffe6b6-bd6d-497a-837e-d747504b4fca admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:44.160077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffe6b6-bd6d-497a-837e-d747504b4fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:44.160579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffe6b6-bd6d-497a-837e-d747504b4fca admin admin] PATH_INFO: `/v3/domains` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:44.161296 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffe6b6-bd6d-497a-837e-d747504b4fca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81616) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:44:44.161932 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffe6b6-bd6d-497a-837e-d747504b4fca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:44.164196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffe6b6-bd6d-497a-837e-d747504b4fca admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:44.175484 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1262/2520] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:44:44.180123 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067d40f-c09e-46a7-9ad9-9693e209458e None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:44.189396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067d40f-c09e-46a7-9ad9-9693e209458e None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:44.193991 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fc005-d1e7-4b55-8897-85aee3f2abba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:44.197353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fc005-d1e7-4b55-8897-85aee3f2abba admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:44.197353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fc005-d1e7-4b55-8897-85aee3f2abba admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:44.197353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fc005-d1e7-4b55-8897-85aee3f2abba admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:44.197353 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fc005-d1e7-4b55-8897-85aee3f2abba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:44.197756 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fc005-d1e7-4b55-8897-85aee3f2abba admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:44.274172 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1259/2521] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:44:44 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:44.278214 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffe6b6-bd6d-497a-837e-d747504b4fca admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:44.287874 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffe6b6-bd6d-497a-837e-d747504b4fca admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:44.345278 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bd22c-6bb8-40a1-8ab8-85fab44a9afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:44.345914 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bd22c-6bb8-40a1-8ab8-85fab44a9afd admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:44.346619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bd22c-6bb8-40a1-8ab8-85fab44a9afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:44.346619 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bd22c-6bb8-40a1-8ab8-85fab44a9afd admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:44.347142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bd22c-6bb8-40a1-8ab8-85fab44a9afd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:44.349091 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bd22c-6bb8-40a1-8ab8-85fab44a9afd admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:44.427806 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1263/2522] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:44:44 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:44.432908 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fc005-d1e7-4b55-8897-85aee3f2abba admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:44.443277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fc005-d1e7-4b55-8897-85aee3f2abba admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:44.448798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a0de2-c9df-4b00-a4ab-5ebd3bc21298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:44.448798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a0de2-c9df-4b00-a4ab-5ebd3bc21298 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:44.448798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a0de2-c9df-4b00-a4ab-5ebd3bc21298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:44.449174 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a0de2-c9df-4b00-a4ab-5ebd3bc21298 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:44.449808 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a0de2-c9df-4b00-a4ab-5ebd3bc21298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:44.451429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a0de2-c9df-4b00-a4ab-5ebd3bc21298 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:44.462032 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1260/2523] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:44.466410 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bd22c-6bb8-40a1-8ab8-85fab44a9afd admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:44.476383 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bd22c-6bb8-40a1-8ab8-85fab44a9afd admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:44.481731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2bd71-cb51-4487-863f-ab24df0ae533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:44.482369 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2bd71-cb51-4487-863f-ab24df0ae533 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:44.482653 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2bd71-cb51-4487-863f-ab24df0ae533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:44.482927 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2bd71-cb51-4487-863f-ab24df0ae533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:44.483613 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2bd71-cb51-4487-863f-ab24df0ae533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:44.485545 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2bd71-cb51-4487-863f-ab24df0ae533 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:44.496326 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1264/2524] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:44.500385 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a0de2-c9df-4b00-a4ab-5ebd3bc21298 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:44.510593 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a0de2-c9df-4b00-a4ab-5ebd3bc21298 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:44.515226 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4568c-677e-42a4-8f50-e84f0c2e310d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:44.515869 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4568c-677e-42a4-8f50-e84f0c2e310d admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:44.516148 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4568c-677e-42a4-8f50-e84f0c2e310d admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:44.516416 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4568c-677e-42a4-8f50-e84f0c2e310d admin admin] PATH_INFO: `/v3/projects/13d7e76725c84fb68cc2b873b0567680/users/921f75721914480b9112a1f1bdfe9ff2/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:44.531140 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4568c-677e-42a4-8f50-e84f0c2e310d admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d7e76725c84fb68cc2b873b0567680, user_id=921f75721914480b9112a1f1bdfe9ff2, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:44.533725 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4568c-677e-42a4-8f50-e84f0c2e310d admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:44.592457 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1261/2525] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:44 2022] PUT /identity/v3/projects/13d7e76725c84fb68cc2b873b0567680/users/921f75721914480b9112a1f1bdfe9ff2/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:44:44.603392 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468169d0-bee3-4984-bf71-5c0d354e3bb9 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:44.603700 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468169d0-bee3-4984-bf71-5c0d354e3bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:44.603971 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468169d0-bee3-4984-bf71-5c0d354e3bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:44.631322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-468169d0-bee3-4984-bf71-5c0d354e3bb9 None None] MFA Rules not processed for user `921f75721914480b9112a1f1bdfe9ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:44.632529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-468169d0-bee3-4984-bf71-5c0d354e3bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:44.695568 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1265/2526] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:44.700624 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4568c-677e-42a4-8f50-e84f0c2e310d admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:44.760937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4568c-677e-42a4-8f50-e84f0c2e310d admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:44.765596 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c29846-7cf0-4dcb-99e4-90c53ab854d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:44.766210 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c29846-7cf0-4dcb-99e4-90c53ab854d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:44.766509 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c29846-7cf0-4dcb-99e4-90c53ab854d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:44.766760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c29846-7cf0-4dcb-99e4-90c53ab854d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:44.767382 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c29846-7cf0-4dcb-99e4-90c53ab854d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:44.769268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c29846-7cf0-4dcb-99e4-90c53ab854d9 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:44.850671 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1262/2527] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:44:44 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:44.855299 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468169d0-bee3-4984-bf71-5c0d354e3bb9 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:44.865790 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468169d0-bee3-4984-bf71-5c0d354e3bb9 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:44.891642 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:44.920590 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff27f8-b701-4611-873d-20eaae377822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:44.921188 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff27f8-b701-4611-873d-20eaae377822 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:44.921467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff27f8-b701-4611-873d-20eaae377822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:44.921738 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff27f8-b701-4611-873d-20eaae377822 admin admin] PATH_INFO: `/v3/users` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:44.922401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff27f8-b701-4611-873d-20eaae377822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:44.924322 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff27f8-b701-4611-873d-20eaae377822 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:45.035614 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1266/2528] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:44:44 2022] POST /identity/v3/users => generated 300 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:44:45.039472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c29846-7cf0-4dcb-99e4-90c53ab854d9 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:45.050286 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c29846-7cf0-4dcb-99e4-90c53ab854d9 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:45.054821 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61275a33-e8a5-47c3-bf35-7e64ac4a3a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:45.055409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61275a33-e8a5-47c3-bf35-7e64ac4a3a7b admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:45.055686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61275a33-e8a5-47c3-bf35-7e64ac4a3a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:45.055973 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61275a33-e8a5-47c3-bf35-7e64ac4a3a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:45.056592 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61275a33-e8a5-47c3-bf35-7e64ac4a3a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:45.058228 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61275a33-e8a5-47c3-bf35-7e64ac4a3a7b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:45.069456 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1263/2529] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:45.073616 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff27f8-b701-4611-873d-20eaae377822 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:45.083449 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff27f8-b701-4611-873d-20eaae377822 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:45.088492 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d92fd2-252e-4808-beee-5a33a7b2db56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:45.089128 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d92fd2-252e-4808-beee-5a33a7b2db56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:45.089401 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d92fd2-252e-4808-beee-5a33a7b2db56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:45.089667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d92fd2-252e-4808-beee-5a33a7b2db56 admin admin] PATH_INFO: `/v3/projects/e83cb36689444975b440b22e9b11954f/users/66fe5a36463d40ff96dd42de463494d2/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:45.109231 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d92fd2-252e-4808-beee-5a33a7b2db56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83cb36689444975b440b22e9b11954f, user_id=66fe5a36463d40ff96dd42de463494d2, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:45.111914 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d92fd2-252e-4808-beee-5a33a7b2db56 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:45.206256 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1267/2530] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:45 2022] PUT /identity/v3/projects/e83cb36689444975b440b22e9b11954f/users/66fe5a36463d40ff96dd42de463494d2/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:45.210326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61275a33-e8a5-47c3-bf35-7e64ac4a3a7b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:45.220641 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61275a33-e8a5-47c3-bf35-7e64ac4a3a7b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:45.275792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487e8c1-524c-4fe8-ad36-2648aa6e13ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:45.276390 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487e8c1-524c-4fe8-ad36-2648aa6e13ca admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:45.276674 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487e8c1-524c-4fe8-ad36-2648aa6e13ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:45.276943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487e8c1-524c-4fe8-ad36-2648aa6e13ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:45.277572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487e8c1-524c-4fe8-ad36-2648aa6e13ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:45.279204 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487e8c1-524c-4fe8-ad36-2648aa6e13ca admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:45.288895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1264/2531] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:45.293626 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d92fd2-252e-4808-beee-5a33a7b2db56 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:45.303803 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d92fd2-252e-4808-beee-5a33a7b2db56 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:45.308617 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a3ee5-57fa-4326-9f0d-4c84d13c1d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:45.309263 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a3ee5-57fa-4326-9f0d-4c84d13c1d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:45.313962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a3ee5-57fa-4326-9f0d-4c84d13c1d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:45.313962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a3ee5-57fa-4326-9f0d-4c84d13c1d1e admin admin] PATH_INFO: `/v3/projects/e83cb36689444975b440b22e9b11954f/users/66fe5a36463d40ff96dd42de463494d2/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:45.313962 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a3ee5-57fa-4326-9f0d-4c84d13c1d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83cb36689444975b440b22e9b11954f, user_id=66fe5a36463d40ff96dd42de463494d2, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:45.314544 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a3ee5-57fa-4326-9f0d-4c84d13c1d1e admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:45.325060 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1268/2532] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:45 2022] PUT /identity/v3/projects/e83cb36689444975b440b22e9b11954f/users/66fe5a36463d40ff96dd42de463494d2/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:45.336323 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006a9d-9d8c-4d77-b6fa-290d993dec25 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:45.336606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006a9d-9d8c-4d77-b6fa-290d993dec25 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:45.336878 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006a9d-9d8c-4d77-b6fa-290d993dec25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:45.345737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:45.364958 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-20006a9d-9d8c-4d77-b6fa-290d993dec25 None None] MFA Rules not processed for user `66fe5a36463d40ff96dd42de463494d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:45.366139 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-20006a9d-9d8c-4d77-b6fa-290d993dec25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:45.431445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1265/2533] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:45 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:45.442594 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a3ee5-57fa-4326-9f0d-4c84d13c1d1e admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:45.444241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-0a6a3ee5-57fa-4326-9f0d-4c84d13c1d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:45.500599 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a3ee5-57fa-4326-9f0d-4c84d13c1d1e admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:45.504190 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3470f-fb64-4deb-b161-1e1a1c578ab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:45.504780 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3470f-fb64-4deb-b161-1e1a1c578ab2 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:45.505306 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3470f-fb64-4deb-b161-1e1a1c578ab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:45.505574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3470f-fb64-4deb-b161-1e1a1c578ab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:45.514846 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3470f-fb64-4deb-b161-1e1a1c578ab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:45.516242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3470f-fb64-4deb-b161-1e1a1c578ab2 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:45.526027 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1269/2534] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:45 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:45.555004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006a9d-9d8c-4d77-b6fa-290d993dec25 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:45.614633 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006a9d-9d8c-4d77-b6fa-290d993dec25 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:45.619187 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:45.619740 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:45.620013 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:45.620277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] PATH_INFO: `/v3/users/921f75721914480b9112a1f1bdfe9ff2` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:45.621376 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] RBAC: Authorizing `identity:delete_user(user_id=921f75721914480b9112a1f1bdfe9ff2)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:45.623289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:45.805811 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '921f75721914480b9112a1f1bdfe9ff2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:45.807248 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '921f75721914480b9112a1f1bdfe9ff2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:45.875185 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '921f75721914480b9112a1f1bdfe9ff2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:45.875754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '921f75721914480b9112a1f1bdfe9ff2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:45.883135 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '921f75721914480b9112a1f1bdfe9ff2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:45.900658 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1266/2535] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:44:45 2022] DELETE /identity/v3/users/921f75721914480b9112a1f1bdfe9ff2 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:45.905109 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b9f4d-9c87-4b5a-8d34-3c01c901b01a None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:45.905440 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b9f4d-9c87-4b5a-8d34-3c01c901b01a None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:45.905715 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b9f4d-9c87-4b5a-8d34-3c01c901b01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:45.924789 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-a71b9f4d-9c87-4b5a-8d34-3c01c901b01a None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:45.925949 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a71b9f4d-9c87-4b5a-8d34-3c01c901b01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:45.983534 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1270/2536] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:45.993150 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:45.994656 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:46.005782 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:46.051812 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e77bd0-31e5-4989-8d97-ed5f0b07592c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:46.055494 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc584a0-671d-4fd0-b2b1-f4f8cf3a6a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:46.056098 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc584a0-671d-4fd0-b2b1-f4f8cf3a6a49 service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:46.056437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc584a0-671d-4fd0-b2b1-f4f8cf3a6a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:46.056891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc584a0-671d-4fd0-b2b1-f4f8cf3a6a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:46.066224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc584a0-671d-4fd0-b2b1-f4f8cf3a6a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:46.067625 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc584a0-671d-4fd0-b2b1-f4f8cf3a6a49 service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:46.077326 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1267/2537] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:46.459573 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b9f4d-9c87-4b5a-8d34-3c01c901b01a None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:46.461013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-a71b9f4d-9c87-4b5a-8d34-3c01c901b01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:46.476014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b9f4d-9c87-4b5a-8d34-3c01c901b01a None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:46.480880 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:46.481631 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:46.481905 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:46.482198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] PATH_INFO: `/v3/projects/13d7e76725c84fb68cc2b873b0567680` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:46.484365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d7e76725c84fb68cc2b873b0567680)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:46.486365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:46.508431 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:46.573184 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d7e76725c84fb68cc2b873b0567680', 'request_id': 'req-7fdd3320-be65-465d-bde1-2f46baa640ce'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:46.579840 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d7e76725c84fb68cc2b873b0567680', 'request_id': 'req-7fdd3320-be65-465d-bde1-2f46baa640ce'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:46.652475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d7e76725c84fb68cc2b873b0567680', 'request_id': 'req-7fdd3320-be65-465d-bde1-2f46baa640ce'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:46.663660 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] The token cache is being invalidate because project 13d7e76725c84fb68cc2b873b0567680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:44:46.663999 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:46.664570 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:46.666132 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1271/2538] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:44:46 2022] DELETE /identity/v3/projects/13d7e76725c84fb68cc2b873b0567680 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:46.670114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc584a0-671d-4fd0-b2b1-f4f8cf3a6a49 service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:46.679693 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc584a0-671d-4fd0-b2b1-f4f8cf3a6a49 service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:46.734957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa9952-999d-4a85-a97f-c724e4aafb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:46.735503 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa9952-999d-4a85-a97f-c724e4aafb49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:46.735781 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa9952-999d-4a85-a97f-c724e4aafb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:46.736058 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa9952-999d-4a85-a97f-c724e4aafb49 admin admin] PATH_INFO: `/v3/users/66fe5a36463d40ff96dd42de463494d2` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:46.737171 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa9952-999d-4a85-a97f-c724e4aafb49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66fe5a36463d40ff96dd42de463494d2)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:46.739122 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa9952-999d-4a85-a97f-c724e4aafb49 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:46.894184 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d0aa9952-999d-4a85-a97f-c724e4aafb49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66fe5a36463d40ff96dd42de463494d2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:46.894828 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d0aa9952-999d-4a85-a97f-c724e4aafb49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66fe5a36463d40ff96dd42de463494d2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:46.949339 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d0aa9952-999d-4a85-a97f-c724e4aafb49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66fe5a36463d40ff96dd42de463494d2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:46.949937 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d0aa9952-999d-4a85-a97f-c724e4aafb49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66fe5a36463d40ff96dd42de463494d2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:46.956878 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-d0aa9952-999d-4a85-a97f-c724e4aafb49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66fe5a36463d40ff96dd42de463494d2'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:46.973690 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1268/2539] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:44:46 2022] DELETE /identity/v3/users/66fe5a36463d40ff96dd42de463494d2 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:47.336731 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:47.338755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:47.351648 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd3320-be65-465d-bde1-2f46baa640ce admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:47.408537 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:47.409165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:47.409442 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:47.409712 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] PATH_INFO: `/v3/projects/e83cb36689444975b440b22e9b11954f` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:47.410842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e83cb36689444975b440b22e9b11954f)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:47.412740 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:47.508487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e83cb36689444975b440b22e9b11954f', 'request_id': 'req-e0f76927-5385-46c2-975c-6b86f1f08649'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:47.514583 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e83cb36689444975b440b22e9b11954f', 'request_id': 'req-e0f76927-5385-46c2-975c-6b86f1f08649'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:47.569252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e83cb36689444975b440b22e9b11954f', 'request_id': 'req-e0f76927-5385-46c2-975c-6b86f1f08649'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:47.580652 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] The token cache is being invalidate because project e83cb36689444975b440b22e9b11954f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:44:47.580929 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:47.581553 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-e0f76927-5385-46c2-975c-6b86f1f08649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:47.583043 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1272/2540] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:44:47 2022] DELETE /identity/v3/projects/e83cb36689444975b440b22e9b11954f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:47.588397 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd8bb8-41da-4157-be61-deec587161b5 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:47.588682 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd8bb8-41da-4157-be61-deec587161b5 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:47.588954 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd8bb8-41da-4157-be61-deec587161b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:47.608933 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e6cd8bb8-41da-4157-be61-deec587161b5 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:47.610143 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e6cd8bb8-41da-4157-be61-deec587161b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:47.636852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:47.670684 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1269/2541] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:47.678486 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a8254-dfd7-4722-a01b-ec9551e72759 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:47.678772 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a8254-dfd7-4722-a01b-ec9551e72759 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:47.679047 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a8254-dfd7-4722-a01b-ec9551e72759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:47.699215 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-ca2a8254-dfd7-4722-a01b-ec9551e72759 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:47.700405 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-ca2a8254-dfd7-4722-a01b-ec9551e72759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:47.708161 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1273/2542] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:47.712346 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd8bb8-41da-4157-be61-deec587161b5 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:47.722529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd8bb8-41da-4157-be61-deec587161b5 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:47.727686 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df82a9-92bf-496f-95e5-e8a7e5abad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:47.728277 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df82a9-92bf-496f-95e5-e8a7e5abad7b admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:47.728557 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df82a9-92bf-496f-95e5-e8a7e5abad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:47.728823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df82a9-92bf-496f-95e5-e8a7e5abad7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:47.729461 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df82a9-92bf-496f-95e5-e8a7e5abad7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:44:47.729754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df82a9-92bf-496f-95e5-e8a7e5abad7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:47.731370 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df82a9-92bf-496f-95e5-e8a7e5abad7b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:47.742442 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1270/2543] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:44:47.746645 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a8254-dfd7-4722-a01b-ec9551e72759 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:47.755973 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a8254-dfd7-4722-a01b-ec9551e72759 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:47.761072 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074932c9-337e-45f6-9433-3b8220faa0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:47.761654 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074932c9-337e-45f6-9433-3b8220faa0b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:47.761938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074932c9-337e-45f6-9433-3b8220faa0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:47.762242 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074932c9-337e-45f6-9433-3b8220faa0b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:47.762884 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074932c9-337e-45f6-9433-3b8220faa0b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:47.764744 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074932c9-337e-45f6-9433-3b8220faa0b3 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:47.844636 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1274/2544] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:44:47 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:47.848660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df82a9-92bf-496f-95e5-e8a7e5abad7b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:47.859246 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df82a9-92bf-496f-95e5-e8a7e5abad7b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:47.918988 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35815c8a-e2ce-4fb9-b460-a0fe7a2dbb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:47.919642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35815c8a-e2ce-4fb9-b460-a0fe7a2dbb02 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:47.919842 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35815c8a-e2ce-4fb9-b460-a0fe7a2dbb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:47.920106 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35815c8a-e2ce-4fb9-b460-a0fe7a2dbb02 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:47.920717 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35815c8a-e2ce-4fb9-b460-a0fe7a2dbb02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:47.923012 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35815c8a-e2ce-4fb9-b460-a0fe7a2dbb02 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:47.992396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1271/2545] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:44:47 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:47.996995 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074932c9-337e-45f6-9433-3b8220faa0b3 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:48.007438 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074932c9-337e-45f6-9433-3b8220faa0b3 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:48.012584 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8238460-4d17-4fd5-b9ff-ee57eed3ec04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:48.013175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8238460-4d17-4fd5-b9ff-ee57eed3ec04 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:48.013448 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8238460-4d17-4fd5-b9ff-ee57eed3ec04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:48.013717 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8238460-4d17-4fd5-b9ff-ee57eed3ec04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:48.014452 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8238460-4d17-4fd5-b9ff-ee57eed3ec04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:48.017532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8238460-4d17-4fd5-b9ff-ee57eed3ec04 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:48.018008 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:48.026089 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1275/2546] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:48.029780 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35815c8a-e2ce-4fb9-b460-a0fe7a2dbb02 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:48.039644 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35815c8a-e2ce-4fb9-b460-a0fe7a2dbb02 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:48.045090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30362491-d2b0-41dd-95e4-62365e253df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:48.045683 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30362491-d2b0-41dd-95e4-62365e253df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:48.045966 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30362491-d2b0-41dd-95e4-62365e253df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:48.046261 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30362491-d2b0-41dd-95e4-62365e253df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:48.046894 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30362491-d2b0-41dd-95e4-62365e253df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:48.048539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30362491-d2b0-41dd-95e4-62365e253df1 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:48.058952 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1272/2547] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:48.063241 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8238460-4d17-4fd5-b9ff-ee57eed3ec04 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:48.073723 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8238460-4d17-4fd5-b9ff-ee57eed3ec04 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:48.078404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df0e4-4054-4be0-97f5-5d3dff2f016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:48.079047 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df0e4-4054-4be0-97f5-5d3dff2f016a admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:48.079379 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df0e4-4054-4be0-97f5-5d3dff2f016a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:48.079684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df0e4-4054-4be0-97f5-5d3dff2f016a admin admin] PATH_INFO: `/v3/projects/b69151739f014ffca2a21f6a0d60e785/users/ae3372dff4a94ded8eb436f39f324707/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:48.100818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093df0e4-4054-4be0-97f5-5d3dff2f016a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b69151739f014ffca2a21f6a0d60e785, user_id=ae3372dff4a94ded8eb436f39f324707, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:48.100818 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093df0e4-4054-4be0-97f5-5d3dff2f016a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:48.181118 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1276/2548] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:48 2022] PUT /identity/v3/projects/b69151739f014ffca2a21f6a0d60e785/users/ae3372dff4a94ded8eb436f39f324707/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:48.193111 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ac6ba-ffa9-4800-b9e4-6c43eb281795 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:48.193393 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ac6ba-ffa9-4800-b9e4-6c43eb281795 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:48.193667 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ac6ba-ffa9-4800-b9e4-6c43eb281795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:48.221833 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-1a7ac6ba-ffa9-4800-b9e4-6c43eb281795 None None] MFA Rules not processed for user `ae3372dff4a94ded8eb436f39f324707`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:48.223074 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1a7ac6ba-ffa9-4800-b9e4-6c43eb281795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:48.285925 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1273/2549] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:48.290435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df0e4-4054-4be0-97f5-5d3dff2f016a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:48.351014 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df0e4-4054-4be0-97f5-5d3dff2f016a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:48.356196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64364f8a-df49-4978-8b2d-9a413ce27df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:48.356794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64364f8a-df49-4978-8b2d-9a413ce27df2 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:48.357082 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64364f8a-df49-4978-8b2d-9a413ce27df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:48.357354 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64364f8a-df49-4978-8b2d-9a413ce27df2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:48.357981 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64364f8a-df49-4978-8b2d-9a413ce27df2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:48.359907 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64364f8a-df49-4978-8b2d-9a413ce27df2 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:48.445441 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1277/2550] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:44:48 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:48.449482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ac6ba-ffa9-4800-b9e4-6c43eb281795 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:48.460757 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ac6ba-ffa9-4800-b9e4-6c43eb281795 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:48.484932 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:48.516974 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52525b2f-ce19-4863-8799-43ebe1080bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:48.517606 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52525b2f-ce19-4863-8799-43ebe1080bca admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:48.517891 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52525b2f-ce19-4863-8799-43ebe1080bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:48.518196 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52525b2f-ce19-4863-8799-43ebe1080bca admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:48.518823 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52525b2f-ce19-4863-8799-43ebe1080bca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:48.520768 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52525b2f-ce19-4863-8799-43ebe1080bca admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:48.635224 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1274/2551] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:44:48 2022] POST /identity/v3/users => generated 290 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:48.639234 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64364f8a-df49-4978-8b2d-9a413ce27df2 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:48.649436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64364f8a-df49-4978-8b2d-9a413ce27df2 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:48.653948 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782a81-7506-4c25-8f09-cc9a415e30f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:48.654603 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782a81-7506-4c25-8f09-cc9a415e30f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:48.654885 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782a81-7506-4c25-8f09-cc9a415e30f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:48.655158 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782a81-7506-4c25-8f09-cc9a415e30f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:48.655796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20782a81-7506-4c25-8f09-cc9a415e30f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:48.657364 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20782a81-7506-4c25-8f09-cc9a415e30f5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:48.668096 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1278/2552] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:48.671930 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52525b2f-ce19-4863-8799-43ebe1080bca admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:48.682429 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52525b2f-ce19-4863-8799-43ebe1080bca admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:48.687077 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdca594-e3c8-44c9-bc3c-1f5563dac1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:48.687668 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdca594-e3c8-44c9-bc3c-1f5563dac1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:48.687948 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdca594-e3c8-44c9-bc3c-1f5563dac1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:48.688222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdca594-e3c8-44c9-bc3c-1f5563dac1c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:48.688861 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdca594-e3c8-44c9-bc3c-1f5563dac1c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:48.690486 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdca594-e3c8-44c9-bc3c-1f5563dac1c8 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:48.700222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1275/2553] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:48.704329 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782a81-7506-4c25-8f09-cc9a415e30f5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:48.714083 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782a81-7506-4c25-8f09-cc9a415e30f5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:48.718683 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac8235-ea9b-4483-9199-5f754ebb83cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:48.719325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac8235-ea9b-4483-9199-5f754ebb83cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:48.719607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac8235-ea9b-4483-9199-5f754ebb83cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:48.719882 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac8235-ea9b-4483-9199-5f754ebb83cf admin admin] PATH_INFO: `/v3/projects/b3ae8d77983b44ca98d424aab7854079/users/596eaeef5bd540cab4043972625fe530/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:48.734677 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac8235-ea9b-4483-9199-5f754ebb83cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ae8d77983b44ca98d424aab7854079, user_id=596eaeef5bd540cab4043972625fe530, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:48.737315 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac8235-ea9b-4483-9199-5f754ebb83cf admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:48.817758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1279/2554] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:48 2022] PUT /identity/v3/projects/b3ae8d77983b44ca98d424aab7854079/users/596eaeef5bd540cab4043972625fe530/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:48.827980 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5203a5c-73ad-4194-9844-a2fbaba92da5 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:48.828265 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5203a5c-73ad-4194-9844-a2fbaba92da5 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:48.828539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5203a5c-73ad-4194-9844-a2fbaba92da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:48.856711 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-b5203a5c-73ad-4194-9844-a2fbaba92da5 None None] MFA Rules not processed for user `596eaeef5bd540cab4043972625fe530`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:48.857913 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-b5203a5c-73ad-4194-9844-a2fbaba92da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:48.920902 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1276/2555] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:48.925280 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac8235-ea9b-4483-9199-5f754ebb83cf admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:48.986164 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac8235-ea9b-4483-9199-5f754ebb83cf admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:48.990804 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4474b43-fda4-431d-b036-a92d3f94651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:48.991404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4474b43-fda4-431d-b036-a92d3f94651d admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:48.991684 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4474b43-fda4-431d-b036-a92d3f94651d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:48.991959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4474b43-fda4-431d-b036-a92d3f94651d admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:48.992595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4474b43-fda4-431d-b036-a92d3f94651d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:48.995103 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4474b43-fda4-431d-b036-a92d3f94651d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:49.079508 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1280/2556] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:44:48 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:44:49.083529 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5203a5c-73ad-4194-9844-a2fbaba92da5 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:49.094512 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5203a5c-73ad-4194-9844-a2fbaba92da5 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:49.153310 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15317eb2-e2ae-46a4-87b8-07b64d8824ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:49.153908 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15317eb2-e2ae-46a4-87b8-07b64d8824ec admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:49.154231 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15317eb2-e2ae-46a4-87b8-07b64d8824ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:49.154491 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15317eb2-e2ae-46a4-87b8-07b64d8824ec admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:49.155126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15317eb2-e2ae-46a4-87b8-07b64d8824ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:49.157025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15317eb2-e2ae-46a4-87b8-07b64d8824ec admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:49.261214 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1277/2557] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:44:49 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:49.265165 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4474b43-fda4-431d-b036-a92d3f94651d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:49.275878 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4474b43-fda4-431d-b036-a92d3f94651d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:49.280435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbac84-0554-474b-bc98-b0e201f55334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:49.281029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbac84-0554-474b-bc98-b0e201f55334 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:49.281307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbac84-0554-474b-bc98-b0e201f55334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:49.281578 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbac84-0554-474b-bc98-b0e201f55334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:49.282287 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbac84-0554-474b-bc98-b0e201f55334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:49.283859 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bbac84-0554-474b-bc98-b0e201f55334 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:49.285860 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:49.295212 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1281/2558] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:49.299235 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15317eb2-e2ae-46a4-87b8-07b64d8824ec admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:49.309249 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15317eb2-e2ae-46a4-87b8-07b64d8824ec admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:49.314005 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8b9e3-7697-49aa-8055-8af48116ed52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:49.316412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8b9e3-7697-49aa-8055-8af48116ed52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:49.316412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8b9e3-7697-49aa-8055-8af48116ed52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:49.316412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db8b9e3-7697-49aa-8055-8af48116ed52 admin admin] PATH_INFO: `/v3/projects/b55f75a9d8b14c71893f481ebce6d474/users/0c20140fe61746b9a7bed21b98741aee/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:49.329440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8b9e3-7697-49aa-8055-8af48116ed52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b55f75a9d8b14c71893f481ebce6d474, user_id=0c20140fe61746b9a7bed21b98741aee, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:49.332119 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db8b9e3-7697-49aa-8055-8af48116ed52 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:49.433949 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:49.438883 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1278/2559] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:49 2022] PUT /identity/v3/projects/b55f75a9d8b14c71893f481ebce6d474/users/0c20140fe61746b9a7bed21b98741aee/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:49.442869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbac84-0554-474b-bc98-b0e201f55334 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:49.453279 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bbac84-0554-474b-bc98-b0e201f55334 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:49.508273 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c373a-1c6b-4cf1-bc53-2b95064b303a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:49.508908 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c373a-1c6b-4cf1-bc53-2b95064b303a admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:49.509187 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c373a-1c6b-4cf1-bc53-2b95064b303a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:49.509460 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c373a-1c6b-4cf1-bc53-2b95064b303a admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:49.510639 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c373a-1c6b-4cf1-bc53-2b95064b303a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:49.512276 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c373a-1c6b-4cf1-bc53-2b95064b303a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:49.521704 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1282/2560] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:49.526552 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8b9e3-7697-49aa-8055-8af48116ed52 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:49.536838 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db8b9e3-7697-49aa-8055-8af48116ed52 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:49.541309 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c46726-3ceb-4e03-ab21-1370fb9a008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:49.541944 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c46726-3ceb-4e03-ab21-1370fb9a008b admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:49.542245 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c46726-3ceb-4e03-ab21-1370fb9a008b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:49.542515 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c46726-3ceb-4e03-ab21-1370fb9a008b admin admin] PATH_INFO: `/v3/projects/b55f75a9d8b14c71893f481ebce6d474/users/0c20140fe61746b9a7bed21b98741aee/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:49.544395 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c46726-3ceb-4e03-ab21-1370fb9a008b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b55f75a9d8b14c71893f481ebce6d474, user_id=0c20140fe61746b9a7bed21b98741aee, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:49.547025 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c46726-3ceb-4e03-ab21-1370fb9a008b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:49.556945 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1279/2561] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:49 2022] PUT /identity/v3/projects/b55f75a9d8b14c71893f481ebce6d474/users/0c20140fe61746b9a7bed21b98741aee/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:49.567087 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603d031-57c7-4def-aeb2-b84ac04ff3ca None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:49.567374 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603d031-57c7-4def-aeb2-b84ac04ff3ca None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:49.567651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603d031-57c7-4def-aeb2-b84ac04ff3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:49.597155 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-c603d031-57c7-4def-aeb2-b84ac04ff3ca None None] MFA Rules not processed for user `0c20140fe61746b9a7bed21b98741aee`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:49.598390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c603d031-57c7-4def-aeb2-b84ac04ff3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:49.663862 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1283/2562] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:49.678481 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c46726-3ceb-4e03-ab21-1370fb9a008b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:49.680967 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c6c46726-3ceb-4e03-ab21-1370fb9a008b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:49.737728 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c46726-3ceb-4e03-ab21-1370fb9a008b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:49.741387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3baa-4aac-41f2-b9b7-b9a0079542fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:49.742014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3baa-4aac-41f2-b9b7-b9a0079542fd service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:49.742324 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3baa-4aac-41f2-b9b7-b9a0079542fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:49.742594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3baa-4aac-41f2-b9b7-b9a0079542fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:49.751478 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d3baa-4aac-41f2-b9b7-b9a0079542fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:49.752927 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d3baa-4aac-41f2-b9b7-b9a0079542fd service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:49.763175 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1280/2563] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:49 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:49.866321 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603d031-57c7-4def-aeb2-b84ac04ff3ca None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:49.867688 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-c603d031-57c7-4def-aeb2-b84ac04ff3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:49.880729 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603d031-57c7-4def-aeb2-b84ac04ff3ca None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:49.884252 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebedf6-e880-4b70-9c19-85b8261c4772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:49.885525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebedf6-e880-4b70-9c19-85b8261c4772 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:49.885801 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebedf6-e880-4b70-9c19-85b8261c4772 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:49.886090 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebedf6-e880-4b70-9c19-85b8261c4772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:49.941001 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebedf6-e880-4b70-9c19-85b8261c4772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:49.942509 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebedf6-e880-4b70-9c19-85b8261c4772 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:49.952283 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1284/2564] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:54.007922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3baa-4aac-41f2-b9b7-b9a0079542fd service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:54.068845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3baa-4aac-41f2-b9b7-b9a0079542fd service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:54.073568 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:54.074217 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:54.074498 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:54.074770 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] PATH_INFO: `/v3/users/ae3372dff4a94ded8eb436f39f324707` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:54.075911 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae3372dff4a94ded8eb436f39f324707)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:54.077854 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:54.268805 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae3372dff4a94ded8eb436f39f324707'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:54.269447 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae3372dff4a94ded8eb436f39f324707'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:54.383298 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae3372dff4a94ded8eb436f39f324707'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:54.383852 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae3372dff4a94ded8eb436f39f324707'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:54.391028 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae3372dff4a94ded8eb436f39f324707'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:54.408926 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1281/2565] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:44:54 2022] DELETE /identity/v3/users/ae3372dff4a94ded8eb436f39f324707 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:54.413432 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736681-b9ef-4974-af82-8f54aeb9c19c None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:54.413930 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736681-b9ef-4974-af82-8f54aeb9c19c None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:54.414068 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63736681-b9ef-4974-af82-8f54aeb9c19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:54.433610 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-63736681-b9ef-4974-af82-8f54aeb9c19c None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:54.434830 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-63736681-b9ef-4974-af82-8f54aeb9c19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:54.446521 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=100 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:54.493755 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1285/2566] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:54.503467 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:54.504849 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:54.516059 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:54.562107 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba59409-17b9-4e35-9a74-72205889dc04 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:54.566199 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad7311-9d76-46b4-b9bf-3f9546526f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:54.566795 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad7311-9d76-46b4-b9bf-3f9546526f6e service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:54.567072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad7311-9d76-46b4-b9bf-3f9546526f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:54.567339 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad7311-9d76-46b4-b9bf-3f9546526f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:54.576147 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad7311-9d76-46b4-b9bf-3f9546526f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:54.577555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad7311-9d76-46b4-b9bf-3f9546526f6e service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:54.586691 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1282/2567] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:44:54.991113 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63736681-b9ef-4974-af82-8f54aeb9c19c None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:54.992595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-63736681-b9ef-4974-af82-8f54aeb9c19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:55.007841 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63736681-b9ef-4974-af82-8f54aeb9c19c None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:55.013743 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:55.018198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:55.018198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:55.018198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] PATH_INFO: `/v3/projects/b69151739f014ffca2a21f6a0d60e785` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:55.018198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b69151739f014ffca2a21f6a0d60e785)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:55.018198 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:55.167667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b69151739f014ffca2a21f6a0d60e785', 'request_id': 'req-08deda7d-0b14-4560-989c-94c88e7c23af'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:55.174025 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b69151739f014ffca2a21f6a0d60e785', 'request_id': 'req-08deda7d-0b14-4560-989c-94c88e7c23af'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:55.240756 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b69151739f014ffca2a21f6a0d60e785', 'request_id': 'req-08deda7d-0b14-4560-989c-94c88e7c23af'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:55.251192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] The token cache is being invalidate because project b69151739f014ffca2a21f6a0d60e785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:44:55.251473 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:55.252077 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:55.253538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1286/2568] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:44:54 2022] DELETE /identity/v3/projects/b69151739f014ffca2a21f6a0d60e785 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:55.257982 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad7311-9d76-46b4-b9bf-3f9546526f6e service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:55.267626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad7311-9d76-46b4-b9bf-3f9546526f6e service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:55.323594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:55.324215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:55.324485 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:55.324752 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] PATH_INFO: `/v3/users/596eaeef5bd540cab4043972625fe530` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:55.325835 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=596eaeef5bd540cab4043972625fe530)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:55.327845 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:55.515078 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '596eaeef5bd540cab4043972625fe530'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:55.515645 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '596eaeef5bd540cab4043972625fe530'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:55.577732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '596eaeef5bd540cab4043972625fe530'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:55.578373 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '596eaeef5bd540cab4043972625fe530'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:55.585773 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '596eaeef5bd540cab4043972625fe530'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:55.603458 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1283/2569] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:44:55 2022] DELETE /identity/v3/users/596eaeef5bd540cab4043972625fe530 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:56.025115 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:56.029135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:56.041643 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08deda7d-0b14-4560-989c-94c88e7c23af admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:56.097845 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:56.098532 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:56.098809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:56.099078 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] PATH_INFO: `/v3/projects/b3ae8d77983b44ca98d424aab7854079` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:56.100202 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ae8d77983b44ca98d424aab7854079)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:56.102179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:56.180086 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:56.184390 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ae8d77983b44ca98d424aab7854079', 'request_id': 'req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:56.190635 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ae8d77983b44ca98d424aab7854079', 'request_id': 'req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:56.263503 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ae8d77983b44ca98d424aab7854079', 'request_id': 'req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:56.274595 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] The token cache is being invalidate because project b3ae8d77983b44ca98d424aab7854079 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:44:56.274883 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:56.275502 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:56.277000 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1287/2570] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:44:56 2022] DELETE /identity/v3/projects/b3ae8d77983b44ca98d424aab7854079 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:56.281269 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:56.282555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:56.295594 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc512a-a34a-4a19-8f41-8dcfcb266701 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:56.323895 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:56.350162 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a344646-c784-4d15-ac45-3dcba017f19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:56.350792 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a344646-c784-4d15-ac45-3dcba017f19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:56.351072 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a344646-c784-4d15-ac45-3dcba017f19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:56.351343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a344646-c784-4d15-ac45-3dcba017f19b admin admin] PATH_INFO: `/v3/users/0c20140fe61746b9a7bed21b98741aee` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:56.352472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a344646-c784-4d15-ac45-3dcba017f19b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c20140fe61746b9a7bed21b98741aee)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:56.354959 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a344646-c784-4d15-ac45-3dcba017f19b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:56.566462 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4a344646-c784-4d15-ac45-3dcba017f19b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c20140fe61746b9a7bed21b98741aee'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:56.567026 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4a344646-c784-4d15-ac45-3dcba017f19b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c20140fe61746b9a7bed21b98741aee'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:56.629041 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4a344646-c784-4d15-ac45-3dcba017f19b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c20140fe61746b9a7bed21b98741aee'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:56.629660 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4a344646-c784-4d15-ac45-3dcba017f19b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c20140fe61746b9a7bed21b98741aee'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:56.637349 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-4a344646-c784-4d15-ac45-3dcba017f19b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c20140fe61746b9a7bed21b98741aee'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:56.655585 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1284/2571] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:44:56 2022] DELETE /identity/v3/users/0c20140fe61746b9a7bed21b98741aee => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:57.145912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:57.151030 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:57.163330 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5fcc8-f2bc-4340-b8ac-27ac95319bb5 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:57.221337 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:57.221990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:57.222303 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:57.222573 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] PATH_INFO: `/v3/projects/b55f75a9d8b14c71893f481ebce6d474` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:57.223664 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b55f75a9d8b14c71893f481ebce6d474)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:57.225591 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:57.329368 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b55f75a9d8b14c71893f481ebce6d474', 'request_id': 'req-c4814455-640c-487a-a6f5-3013ac77ed33'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:57.335527 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b55f75a9d8b14c71893f481ebce6d474', 'request_id': 'req-c4814455-640c-487a-a6f5-3013ac77ed33'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:57.406112 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b55f75a9d8b14c71893f481ebce6d474', 'request_id': 'req-c4814455-640c-487a-a6f5-3013ac77ed33'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:57.416475 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] The token cache is being invalidate because project b55f75a9d8b14c71893f481ebce6d474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:44:57.418404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:57.418404 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c4814455-640c-487a-a6f5-3013ac77ed33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:44:57.418875 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1288/2572] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:44:57 2022] DELETE /identity/v3/projects/b55f75a9d8b14c71893f481ebce6d474 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:57.424766 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b286e7-7e0a-44af-b6e6-90fc6a0041f8 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:57.425240 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b286e7-7e0a-44af-b6e6-90fc6a0041f8 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:57.425344 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b286e7-7e0a-44af-b6e6-90fc6a0041f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:57.445212 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-e7b286e7-7e0a-44af-b6e6-90fc6a0041f8 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:57.446430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-e7b286e7-7e0a-44af-b6e6-90fc6a0041f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:57.503961 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1285/2573] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:57.514359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272773c-a962-4cf7-9228-56afd2753fa0 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:57.514641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272773c-a962-4cf7-9228-56afd2753fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:57.514912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272773c-a962-4cf7-9228-56afd2753fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:57.534179 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-7272773c-a962-4cf7-9228-56afd2753fa0 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:57.535365 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-7272773c-a962-4cf7-9228-56afd2753fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:57.543043 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1289/2574] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:57.547225 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b286e7-7e0a-44af-b6e6-90fc6a0041f8 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:57.556556 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b286e7-7e0a-44af-b6e6-90fc6a0041f8 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:57.561525 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189a023-9643-4ad5-8f77-199822464353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:57.562168 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189a023-9643-4ad5-8f77-199822464353 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:57.562443 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189a023-9643-4ad5-8f77-199822464353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:57.562711 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189a023-9643-4ad5-8f77-199822464353 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:57.563343 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189a023-9643-4ad5-8f77-199822464353 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:44:57.563642 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189a023-9643-4ad5-8f77-199822464353 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:57.565220 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189a023-9643-4ad5-8f77-199822464353 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:57.576869 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1286/2575] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:44:57.581074 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272773c-a962-4cf7-9228-56afd2753fa0 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:57.590333 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272773c-a962-4cf7-9228-56afd2753fa0 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:57.595013 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeb56b-f418-4a4f-8d4b-a65d6efdb3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:57.595604 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeb56b-f418-4a4f-8d4b-a65d6efdb3a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:57.595890 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeb56b-f418-4a4f-8d4b-a65d6efdb3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:57.596161 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaeb56b-f418-4a4f-8d4b-a65d6efdb3a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:57.596794 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaeb56b-f418-4a4f-8d4b-a65d6efdb3a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:57.598721 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaeb56b-f418-4a4f-8d4b-a65d6efdb3a7 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:57.679554 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1290/2576] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:44:57 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:57.684073 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189a023-9643-4ad5-8f77-199822464353 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:57.694004 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189a023-9643-4ad5-8f77-199822464353 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:57.734354 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:57.757555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f500e3-dffd-42e8-a703-e1589fa5c864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:57.758180 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f500e3-dffd-42e8-a703-e1589fa5c864 admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:57.758460 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f500e3-dffd-42e8-a703-e1589fa5c864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:57.758729 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f500e3-dffd-42e8-a703-e1589fa5c864 admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:57.759361 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f500e3-dffd-42e8-a703-e1589fa5c864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:57.761263 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f500e3-dffd-42e8-a703-e1589fa5c864 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:57.863753 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1287/2577] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:44:57 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:44:57.868075 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeb56b-f418-4a4f-8d4b-a65d6efdb3a7 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:57.878589 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaeb56b-f418-4a4f-8d4b-a65d6efdb3a7 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:57.883569 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023d8bd-8c3a-4635-b8c1-8471e912c15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:57.886523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d8bd-8c3a-4635-b8c1-8471e912c15d admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:57.886523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d8bd-8c3a-4635-b8c1-8471e912c15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:57.886523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023d8bd-8c3a-4635-b8c1-8471e912c15d admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:57.886523 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023d8bd-8c3a-4635-b8c1-8471e912c15d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:57.887762 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023d8bd-8c3a-4635-b8c1-8471e912c15d admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:57.897796 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1291/2578] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:57.902422 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f500e3-dffd-42e8-a703-e1589fa5c864 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:57.912340 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f500e3-dffd-42e8-a703-e1589fa5c864 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:57.917302 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510e57d-21d3-4d92-876f-083c41e9a4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:57.917897 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510e57d-21d3-4d92-876f-083c41e9a4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:57.918209 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510e57d-21d3-4d92-876f-083c41e9a4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:57.918482 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510e57d-21d3-4d92-876f-083c41e9a4d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:57.919114 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510e57d-21d3-4d92-876f-083c41e9a4d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:57.920700 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510e57d-21d3-4d92-876f-083c41e9a4d2 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:57.930743 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1288/2579] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:44:57.934658 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023d8bd-8c3a-4635-b8c1-8471e912c15d admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:57.944501 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023d8bd-8c3a-4635-b8c1-8471e912c15d admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:57.949600 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db7ff3-dfb2-4920-b804-b42600ddb1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:57.950253 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7ff3-dfb2-4920-b804-b42600ddb1c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:57.950529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7ff3-dfb2-4920-b804-b42600ddb1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:57.950798 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db7ff3-dfb2-4920-b804-b42600ddb1c9 admin admin] PATH_INFO: `/v3/projects/c0138642e7fb47d68cfc7f951e9d3add/users/7f7a4637c65d47ae9f4ba28d2a6ff98b/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:57.965615 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db7ff3-dfb2-4920-b804-b42600ddb1c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0138642e7fb47d68cfc7f951e9d3add, user_id=7f7a4637c65d47ae9f4ba28d2a6ff98b, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:57.968246 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db7ff3-dfb2-4920-b804-b42600ddb1c9 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:58.055896 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:58.061630 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1292/2580] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:57 2022] PUT /identity/v3/projects/c0138642e7fb47d68cfc7f951e9d3add/users/7f7a4637c65d47ae9f4ba28d2a6ff98b/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:44:58.072818 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989eeff-3823-4737-95bc-4c1a9e0746f2 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:58.073139 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989eeff-3823-4737-95bc-4c1a9e0746f2 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:58.073962 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989eeff-3823-4737-95bc-4c1a9e0746f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:58.101396 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-7989eeff-3823-4737-95bc-4c1a9e0746f2 None None] MFA Rules not processed for user `7f7a4637c65d47ae9f4ba28d2a6ff98b`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:58.102667 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-7989eeff-3823-4737-95bc-4c1a9e0746f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:58.165205 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1289/2581] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:58.169758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db7ff3-dfb2-4920-b804-b42600ddb1c9 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:58.232843 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db7ff3-dfb2-4920-b804-b42600ddb1c9 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:58.237325 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a1e42-a639-40a4-b5d6-4d4df73d0048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:58.237925 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a1e42-a639-40a4-b5d6-4d4df73d0048 admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:58.238235 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a1e42-a639-40a4-b5d6-4d4df73d0048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:58.238518 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a1e42-a639-40a4-b5d6-4d4df73d0048 admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:58.239157 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a1e42-a639-40a4-b5d6-4d4df73d0048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:58.241192 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a1e42-a639-40a4-b5d6-4d4df73d0048 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:58.348574 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1293/2582] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:44:58 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:44:58.352762 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989eeff-3823-4737-95bc-4c1a9e0746f2 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:58.363342 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989eeff-3823-4737-95bc-4c1a9e0746f2 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:58.422657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88c818-fbe2-4201-923e-bcf0923b975b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:58.423266 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88c818-fbe2-4201-923e-bcf0923b975b admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:58.423544 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88c818-fbe2-4201-923e-bcf0923b975b admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:58.423816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a88c818-fbe2-4201-923e-bcf0923b975b admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:58.424462 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88c818-fbe2-4201-923e-bcf0923b975b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:58.426455 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a88c818-fbe2-4201-923e-bcf0923b975b admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:58.583900 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1290/2583] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:44:58 2022] POST /identity/v3/users => generated 297 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 19 02:44:58.588259 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a1e42-a639-40a4-b5d6-4d4df73d0048 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:58.598341 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a1e42-a639-40a4-b5d6-4d4df73d0048 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:58.604203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131af3d-473c-4a33-b6fa-e2c01ff9a197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:58.604203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131af3d-473c-4a33-b6fa-e2c01ff9a197 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:58.604203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131af3d-473c-4a33-b6fa-e2c01ff9a197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:58.604203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131af3d-473c-4a33-b6fa-e2c01ff9a197 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:58.605081 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131af3d-473c-4a33-b6fa-e2c01ff9a197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:58.606869 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131af3d-473c-4a33-b6fa-e2c01ff9a197 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:58.617045 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1294/2584] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:44:58.621903 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88c818-fbe2-4201-923e-bcf0923b975b admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:58.632014 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a88c818-fbe2-4201-923e-bcf0923b975b admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:58.636598 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183af07-d0de-4626-bf2e-05f0404d5231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:58.637237 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183af07-d0de-4626-bf2e-05f0404d5231 admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:58.637531 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183af07-d0de-4626-bf2e-05f0404d5231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:58.637788 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183af07-d0de-4626-bf2e-05f0404d5231 admin admin] PATH_INFO: `/v3/projects/46242be3c92c40f1ad25d769ee001860/users/d0a1259c4036463d868d1efdaf4dd6cf/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:58.653270 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183af07-d0de-4626-bf2e-05f0404d5231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46242be3c92c40f1ad25d769ee001860, user_id=d0a1259c4036463d868d1efdaf4dd6cf, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:58.656000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183af07-d0de-4626-bf2e-05f0404d5231 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:58.725920 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1291/2585] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:58 2022] PUT /identity/v3/projects/46242be3c92c40f1ad25d769ee001860/users/d0a1259c4036463d868d1efdaf4dd6cf/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:58.730581 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131af3d-473c-4a33-b6fa-e2c01ff9a197 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:58.740436 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131af3d-473c-4a33-b6fa-e2c01ff9a197 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:58.794500 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a61d91-dc46-4ad5-ab34-c2f35ed50f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:58.795094 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a61d91-dc46-4ad5-ab34-c2f35ed50f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:58.795370 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a61d91-dc46-4ad5-ab34-c2f35ed50f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:58.795641 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a61d91-dc46-4ad5-ab34-c2f35ed50f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:58.796269 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a61d91-dc46-4ad5-ab34-c2f35ed50f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:58.797856 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a61d91-dc46-4ad5-ab34-c2f35ed50f14 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:58.807207 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1295/2586] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 19 02:44:58.812049 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183af07-d0de-4626-bf2e-05f0404d5231 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:58.822084 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183af07-d0de-4626-bf2e-05f0404d5231 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:58.826604 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222552d-944b-4c2d-95cd-e68dd75118ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:58.827239 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222552d-944b-4c2d-95cd-e68dd75118ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:58.827511 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222552d-944b-4c2d-95cd-e68dd75118ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:58.827776 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222552d-944b-4c2d-95cd-e68dd75118ad admin admin] PATH_INFO: `/v3/projects/46242be3c92c40f1ad25d769ee001860/users/d0a1259c4036463d868d1efdaf4dd6cf/roles/d5070bb151d343ca8de32abe373f4a73` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:58.829626 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222552d-944b-4c2d-95cd-e68dd75118ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=46242be3c92c40f1ad25d769ee001860, user_id=d0a1259c4036463d868d1efdaf4dd6cf, role_id=d5070bb151d343ca8de32abe373f4a73)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:58.832253 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222552d-944b-4c2d-95cd-e68dd75118ad admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:58.837614 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:58.843428 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1292/2587] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:44:58 2022] PUT /identity/v3/projects/46242be3c92c40f1ad25d769ee001860/users/d0a1259c4036463d868d1efdaf4dd6cf/roles/d5070bb151d343ca8de32abe373f4a73 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:44:58.858295 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0660-a794-4bf0-b720-22b2c792768f None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:58.858579 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0660-a794-4bf0-b720-22b2c792768f None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:58.858851 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0660-a794-4bf0-b720-22b2c792768f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:58.886141 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-909b0660-a794-4bf0-b720-22b2c792768f None None] MFA Rules not processed for user `d0a1259c4036463d868d1efdaf4dd6cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:44:58.887314 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-909b0660-a794-4bf0-b720-22b2c792768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:58.944597 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:44:58.954481 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1296/2588] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:44:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:44:58.966239 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222552d-944b-4c2d-95cd-e68dd75118ad admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:44:58.967943 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-6222552d-944b-4c2d-95cd-e68dd75118ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:44:59.024268 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222552d-944b-4c2d-95cd-e68dd75118ad admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:44:59.028430 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51759557-fe7b-4200-93b1-74c32faba972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:44:59.029018 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51759557-fe7b-4200-93b1-74c32faba972 service cinder] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:44:59.029289 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51759557-fe7b-4200-93b1-74c32faba972 service cinder] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:44:59.029555 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51759557-fe7b-4200-93b1-74c32faba972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:44:59.082250 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51759557-fe7b-4200-93b1-74c32faba972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:44:59.083657 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51759557-fe7b-4200-93b1-74c32faba972 service cinder] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:44:59.092797 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1293/2589] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:44:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:45:04.446842 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b0660-a794-4bf0-b720-22b2c792768f None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:04.448906 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-909b0660-a794-4bf0-b720-22b2c792768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:04.460458 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b0660-a794-4bf0-b720-22b2c792768f None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:04.463936 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e903a4-44a8-416e-81ba-9250ec789290 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:04.464525 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e903a4-44a8-416e-81ba-9250ec789290 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:04.464799 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e903a4-44a8-416e-81ba-9250ec789290 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:04.465067 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e903a4-44a8-416e-81ba-9250ec789290 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:04.473709 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e903a4-44a8-416e-81ba-9250ec789290 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:04.475142 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e903a4-44a8-416e-81ba-9250ec789290 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:04.484853 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1297/2590] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:45:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:45:09.088387 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=49 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:45:10.416136 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51759557-fe7b-4200-93b1-74c32faba972 service cinder] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:10.477334 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51759557-fe7b-4200-93b1-74c32faba972 service cinder] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:10.481876 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:10.482521 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:10.482803 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:10.483081 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] PATH_INFO: `/v3/users/7f7a4637c65d47ae9f4ba28d2a6ff98b` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:10.484181 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7a4637c65d47ae9f4ba28d2a6ff98b)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:10.486120 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:10.671722 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7a4637c65d47ae9f4ba28d2a6ff98b'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:10.672306 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7a4637c65d47ae9f4ba28d2a6ff98b'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:10.749091 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7a4637c65d47ae9f4ba28d2a6ff98b'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:10.749700 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7a4637c65d47ae9f4ba28d2a6ff98b'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:10.756988 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7a4637c65d47ae9f4ba28d2a6ff98b'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:10.774061 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1294/2591] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:45:10 2022] DELETE /identity/v3/users/7f7a4637c65d47ae9f4ba28d2a6ff98b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:45:10.779007 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d4a83-3cf0-4332-8e4d-80e8c7a9bae6 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:10.779491 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d4a83-3cf0-4332-8e4d-80e8c7a9bae6 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:10.779606 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d4a83-3cf0-4332-8e4d-80e8c7a9bae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:10.798750 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-e23d4a83-3cf0-4332-8e4d-80e8c7a9bae6 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:45:10.799937 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e23d4a83-3cf0-4332-8e4d-80e8c7a9bae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:10.857203 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1298/2592] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:45:10.866982 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:10.868401 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:10.925322 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf50fb-1647-4522-8c09-7a4a23d79f2c admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:10.928824 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba23c5-ed66-46b6-bb68-9b2b9960abfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:10.929435 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba23c5-ed66-46b6-bb68-9b2b9960abfd service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:10.929713 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba23c5-ed66-46b6-bb68-9b2b9960abfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:10.929987 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba23c5-ed66-46b6-bb68-9b2b9960abfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:10.938839 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba23c5-ed66-46b6-bb68-9b2b9960abfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:10.940272 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba23c5-ed66-46b6-bb68-9b2b9960abfd service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:10.950069 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1295/2593] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:45:11.336776 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d4a83-3cf0-4332-8e4d-80e8c7a9bae6 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:11.338205 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-e23d4a83-3cf0-4332-8e4d-80e8c7a9bae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:11.349912 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d4a83-3cf0-4332-8e4d-80e8c7a9bae6 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:11.354651 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:11.355271 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:11.358073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:11.358073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] PATH_INFO: `/v3/projects/c0138642e7fb47d68cfc7f951e9d3add` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:11.358073 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0138642e7fb47d68cfc7f951e9d3add)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:11.358985 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:11.472667 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0138642e7fb47d68cfc7f951e9d3add', 'request_id': 'req-667a4ae2-f1da-4f20-8be0-74a036ed5233'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:11.478871 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0138642e7fb47d68cfc7f951e9d3add', 'request_id': 'req-667a4ae2-f1da-4f20-8be0-74a036ed5233'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:11.564359 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0138642e7fb47d68cfc7f951e9d3add', 'request_id': 'req-667a4ae2-f1da-4f20-8be0-74a036ed5233'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:11.575023 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] The token cache is being invalidate because project c0138642e7fb47d68cfc7f951e9d3add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:45:11.575307 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:11.580175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:11.580175 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1299/2594] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:45:11 2022] DELETE /identity/v3/projects/c0138642e7fb47d68cfc7f951e9d3add => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:45:11.581587 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba23c5-ed66-46b6-bb68-9b2b9960abfd service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:11.591186 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba23c5-ed66-46b6-bb68-9b2b9960abfd service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:11.609251 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:45:11.646321 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cec6f9-287e-4649-a665-c891138ec0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:11.646939 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cec6f9-287e-4649-a665-c891138ec0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:11.647215 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cec6f9-287e-4649-a665-c891138ec0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:11.647484 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cec6f9-287e-4649-a665-c891138ec0a7 admin admin] PATH_INFO: `/v3/users/d0a1259c4036463d868d1efdaf4dd6cf` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:11.648692 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cec6f9-287e-4649-a665-c891138ec0a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a1259c4036463d868d1efdaf4dd6cf)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:11.650978 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cec6f9-287e-4649-a665-c891138ec0a7 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:11.820200 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-e1cec6f9-287e-4649-a665-c891138ec0a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a1259c4036463d868d1efdaf4dd6cf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:11.820832 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-e1cec6f9-287e-4649-a665-c891138ec0a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a1259c4036463d868d1efdaf4dd6cf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:11.886381 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-e1cec6f9-287e-4649-a665-c891138ec0a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a1259c4036463d868d1efdaf4dd6cf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:11.887000 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-e1cec6f9-287e-4649-a665-c891138ec0a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a1259c4036463d868d1efdaf4dd6cf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:11.894126 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-e1cec6f9-287e-4649-a665-c891138ec0a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a1259c4036463d868d1efdaf4dd6cf'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:11.911572 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1296/2595] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:45:11 2022] DELETE /identity/v3/users/d0a1259c4036463d868d1efdaf4dd6cf => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 19 02:45:12.287758 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:12.289256 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:12.303538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a4ae2-f1da-4f20-8be0-74a036ed5233 admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:12.360369 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:12.360926 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:12.361200 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:12.361467 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] PATH_INFO: `/v3/projects/46242be3c92c40f1ad25d769ee001860` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:12.362592 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46242be3c92c40f1ad25d769ee001860)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:12.364465 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:12.370479 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:45:12.483959 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46242be3c92c40f1ad25d769ee001860', 'request_id': 'req-059b06e0-e901-47c6-9c6f-2df486aefe94'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:12.490222 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46242be3c92c40f1ad25d769ee001860', 'request_id': 'req-059b06e0-e901-47c6-9c6f-2df486aefe94'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:12.552538 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46242be3c92c40f1ad25d769ee001860', 'request_id': 'req-059b06e0-e901-47c6-9c6f-2df486aefe94'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:12.563018 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] The token cache is being invalidate because project 46242be3c92c40f1ad25d769ee001860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:45:12.563513 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:12.565220 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-059b06e0-e901-47c6-9c6f-2df486aefe94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:12.566230 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1300/2596] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:45:12 2022] DELETE /identity/v3/projects/46242be3c92c40f1ad25d769ee001860 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:45:12.572238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f4257-d6c5-4cc1-868c-b900ab47e8aa None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:12.572528 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f4257-d6c5-4cc1-868c-b900ab47e8aa None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:12.572805 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f4257-d6c5-4cc1-868c-b900ab47e8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:12.593132 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-0a5f4257-d6c5-4cc1-868c-b900ab47e8aa None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:45:12.594380 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-0a5f4257-d6c5-4cc1-868c-b900ab47e8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:12.651412 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1297/2597] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:45:12.661177 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22335737-8099-4bbc-9d3c-527392afae77 None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:12.661487 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22335737-8099-4bbc-9d3c-527392afae77 None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:12.661783 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22335737-8099-4bbc-9d3c-527392afae77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:12.681003 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-22335737-8099-4bbc-9d3c-527392afae77 None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:45:12.682250 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-22335737-8099-4bbc-9d3c-527392afae77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:12.690513 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1301/2598] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:45:12.694583 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f4257-d6c5-4cc1-868c-b900ab47e8aa None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:12.704666 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f4257-d6c5-4cc1-868c-b900ab47e8aa None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:12.709615 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452d0e1-2b49-48f2-ac19-ae228c8fbf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:12.710260 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452d0e1-2b49-48f2-ac19-ae228c8fbf11 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:12.710533 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452d0e1-2b49-48f2-ac19-ae228c8fbf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:12.710798 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452d0e1-2b49-48f2-ac19-ae228c8fbf11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:12.711440 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452d0e1-2b49-48f2-ac19-ae228c8fbf11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81615) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 19 02:45:12.711754 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452d0e1-2b49-48f2-ac19-ae228c8fbf11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:12.713338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452d0e1-2b49-48f2-ac19-ae228c8fbf11 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:12.725001 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1298/2599] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Oct 19 02:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 19 02:45:12.731817 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22335737-8099-4bbc-9d3c-527392afae77 None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:12.741080 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22335737-8099-4bbc-9d3c-527392afae77 None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:12.745670 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980adcb-0389-43db-bc6e-684766bb892a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:12.746809 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980adcb-0389-43db-bc6e-684766bb892a admin admin] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:12.747084 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980adcb-0389-43db-bc6e-684766bb892a admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:12.747355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980adcb-0389-43db-bc6e-684766bb892a admin admin] PATH_INFO: `/v3/projects` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:12.748035 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980adcb-0389-43db-bc6e-684766bb892a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:12.749943 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980adcb-0389-43db-bc6e-684766bb892a admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:12.828691 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1302/2600] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Oct 19 02:45:12 2022] POST /identity/v3/projects => generated 359 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:45:12.832889 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452d0e1-2b49-48f2-ac19-ae228c8fbf11 admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:12.842712 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452d0e1-2b49-48f2-ac19-ae228c8fbf11 admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:12.867433 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:45:12.899191 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505c0da-c402-4f5e-a982-294e76f3beaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:12.899869 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505c0da-c402-4f5e-a982-294e76f3beaa admin admin] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:12.900149 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505c0da-c402-4f5e-a982-294e76f3beaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:12.900415 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505c0da-c402-4f5e-a982-294e76f3beaa admin admin] PATH_INFO: `/v3/users` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:12.901057 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505c0da-c402-4f5e-a982-294e76f3beaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:12.903054 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505c0da-c402-4f5e-a982-294e76f3beaa admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:13.015870 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1299/2601] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Oct 19 02:45:12 2022] POST /identity/v3/users => generated 280 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 19 02:45:13.019893 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980adcb-0389-43db-bc6e-684766bb892a admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:13.029938 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980adcb-0389-43db-bc6e-684766bb892a admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:13.034572 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50757a-b45d-43b6-bbca-d0f7b0d642ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:13.035169 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50757a-b45d-43b6-bbca-d0f7b0d642ec admin admin] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:13.035450 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50757a-b45d-43b6-bbca-d0f7b0d642ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:13.036457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50757a-b45d-43b6-bbca-d0f7b0d642ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:13.036457 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50757a-b45d-43b6-bbca-d0f7b0d642ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:13.037939 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50757a-b45d-43b6-bbca-d0f7b0d642ec admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:13.048108 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1303/2602] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:45:13.051908 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505c0da-c402-4f5e-a982-294e76f3beaa admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:13.062955 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505c0da-c402-4f5e-a982-294e76f3beaa admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:13.067565 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108c57e3-c1c3-4d2f-b0c5-9a5cae5f93a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:13.068159 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c57e3-c1c3-4d2f-b0c5-9a5cae5f93a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:13.068448 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c57e3-c1c3-4d2f-b0c5-9a5cae5f93a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:13.068737 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108c57e3-c1c3-4d2f-b0c5-9a5cae5f93a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:13.069362 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c57e3-c1c3-4d2f-b0c5-9a5cae5f93a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:13.070957 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108c57e3-c1c3-4d2f-b0c5-9a5cae5f93a4 admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:13.080648 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1300/2603] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Oct 19 02:45:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 19 02:45:13.086211 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50757a-b45d-43b6-bbca-d0f7b0d642ec admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:13.096029 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50757a-b45d-43b6-bbca-d0f7b0d642ec admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:13.100911 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f1f98-3878-457f-82bb-30b2106a61ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:13.101563 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f1f98-3878-457f-82bb-30b2106a61ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:13.101887 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f1f98-3878-457f-82bb-30b2106a61ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:13.102183 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f1f98-3878-457f-82bb-30b2106a61ed admin admin] PATH_INFO: `/v3/projects/f8397e5b50cd42a385b9c5e9ccfeeb4a/users/84aafe6b697d41818e90803df451e4ef/roles/bb36a823b2ff45d29018aedf19ae7bf0` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:13.117529 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f1f98-3878-457f-82bb-30b2106a61ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8397e5b50cd42a385b9c5e9ccfeeb4a, user_id=84aafe6b697d41818e90803df451e4ef, role_id=bb36a823b2ff45d29018aedf19ae7bf0)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:13.121946 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f1f98-3878-457f-82bb-30b2106a61ed admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:13.200309 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1304/2604] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Oct 19 02:45:13 2022] PUT /identity/v3/projects/f8397e5b50cd42a385b9c5e9ccfeeb4a/users/84aafe6b697d41818e90803df451e4ef/roles/bb36a823b2ff45d29018aedf19ae7bf0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:45:13.211732 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaadb91-68ab-4bef-95c8-109769b731e6 None None] REQUEST_METHOD: `POST` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:13.212054 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaadb91-68ab-4bef-95c8-109769b731e6 None None] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:13.212329 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaadb91-68ab-4bef-95c8-109769b731e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:13.239894 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.auth.core [None req-ceaadb91-68ab-4bef-95c8-109769b731e6 None None] MFA Rules not processed for user `84aafe6b697d41818e90803df451e4ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81615) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:45:13.241090 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-ceaadb91-68ab-4bef-95c8-109769b731e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:13.304129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1301/2605] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:45:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 19 02:45:13.338271 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f1f98-3878-457f-82bb-30b2106a61ed admin admin] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:13.340031 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-566f1f98-3878-457f-82bb-30b2106a61ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:13.397135 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f1f98-3878-457f-82bb-30b2106a61ed admin admin] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:13.400726 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68839e-95f9-455d-b488-63e5e2bb3728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e63c5bd8c4264db697947f025c67f512', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:13.401355 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68839e-95f9-455d-b488-63e5e2bb3728 service cinder] REQUEST_METHOD: `GET` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:13.401633 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68839e-95f9-455d-b488-63e5e2bb3728 service cinder] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:13.401910 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68839e-95f9-455d-b488-63e5e2bb3728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:13.410196 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68839e-95f9-455d-b488-63e5e2bb3728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:13.417781 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68839e-95f9-455d-b488-63e5e2bb3728 service cinder] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:13.420737 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1305/2606] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:45:13 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:45:13.438514 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaadb91-68ab-4bef-95c8-109769b731e6 None None] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:13.499437 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaadb91-68ab-4bef-95c8-109769b731e6 None None] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:13.503922 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:13.504539 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:13.504816 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:13.505085 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] PATH_INFO: `/v3/users/84aafe6b697d41818e90803df451e4ef` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:13.506487 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] RBAC: Authorizing `identity:delete_user(user_id=84aafe6b697d41818e90803df451e4ef)` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:13.508388 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:13.678008 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:45:13.683906 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84aafe6b697d41818e90803df451e4ef'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:13.684629 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84aafe6b697d41818e90803df451e4ef'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:13.737472 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84aafe6b697d41818e90803df451e4ef'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:13.738129 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84aafe6b697d41818e90803df451e4ef'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:13.745338 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.notifications [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84aafe6b697d41818e90803df451e4ef'} {{(pid=81615) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:13.767206 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1302/2607] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Oct 19 02:45:13 2022] DELETE /identity/v3/users/84aafe6b697d41818e90803df451e4ef => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:45:13.768251 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f71a35-bb1f-46ee-a535-b65b0105cdad None None] REQUEST_METHOD: `POST` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:13.768567 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f71a35-bb1f-46ee-a535-b65b0105cdad None None] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:13.768840 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f71a35-bb1f-46ee-a535-b65b0105cdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:13.788974 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.auth.core [None req-53f71a35-bb1f-46ee-a535-b65b0105cdad None None] MFA Rules not processed for user `88e388dc7d10482d8944dbc5c7039f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81616) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 19 02:45:13.790201 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-53f71a35-bb1f-46ee-a535-b65b0105cdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:13.818421 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:45:13.848012 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1306/2608] 10.222.0.20 () {58 vars in 958 bytes} [Wed Oct 19 02:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 19 02:45:13.857724 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:13.859409 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:13.915508 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bc9e8-90f2-400b-8152-b64419c6144f admin admin] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:13.919238 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fba69-47ac-44ce-a863-f1f1409d78ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5ff8583997843258e315413abac172f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:13.919837 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fba69-47ac-44ce-a863-f1f1409d78ef service neutron] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:13.920110 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fba69-47ac-44ce-a863-f1f1409d78ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:13.920375 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fba69-47ac-44ce-a863-f1f1409d78ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:13.929222 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fba69-47ac-44ce-a863-f1f1409d78ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:13.930666 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fba69-47ac-44ce-a863-f1f1409d78ef service neutron] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:13.939968 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1303/2609] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:45:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:45:14.362435 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f71a35-bb1f-46ee-a535-b65b0105cdad None None] Authenticating user token {{(pid=81616) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:14.364009 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.fernet_utils [None req-53f71a35-bb1f-46ee-a535-b65b0105cdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81616) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:14.376751 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f71a35-bb1f-46ee-a535-b65b0105cdad None None] Validating token access rules against request {{(pid=81616) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:14.381808 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88e388dc7d10482d8944dbc5c7039f97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '984205d302cd4aa3b16c6c1e0788c92c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81616) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:14.382607 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:14.382888 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:14.388990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] PATH_INFO: `/v3/projects/f8397e5b50cd42a385b9c5e9ccfeeb4a` {{(pid=81616) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:14.388990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8397e5b50cd42a385b9c5e9ccfeeb4a)` {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:14.388990 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] RBAC: Authorization granted {{(pid=81616) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:14.559021 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8397e5b50cd42a385b9c5e9ccfeeb4a', 'request_id': 'req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:14.565219 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8397e5b50cd42a385b9c5e9ccfeeb4a', 'request_id': 'req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:14.638720 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8397e5b50cd42a385b9c5e9ccfeeb4a', 'request_id': 'req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594'} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:14.649544 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] The token cache is being invalidate because project f8397e5b50cd42a385b9c5e9ccfeeb4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81616) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 19 02:45:14.649828 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:14.650470 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG keystone.notifications [None req-c8b2d9e3-3aa2-464f-aa93-965d6ff36594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81616) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 19 02:45:14.656785 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: [pid: 81616|app: 0|req: 1307/2610] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Oct 19 02:45:14 2022] DELETE /identity/v3/projects/f8397e5b50cd42a385b9c5e9ccfeeb4a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 19 02:45:23.935765 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:DELETE=5,keystone:SELECT=40,keystone:INSERT=1 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:45:24.646343 dv-cinder-861491-4-iscsi devstack@keystone.service[81616]: DEBUG dbcounter [-] [81616] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81616) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 02:45:36.583164 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fba69-47ac-44ce-a863-f1f1409d78ef service neutron] Authenticating user token {{(pid=81615) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 19 02:45:36.584548 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-c20fba69-47ac-44ce-a863-f1f1409d78ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:36.652311 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fba69-47ac-44ce-a863-f1f1409d78ef service neutron] Validating token access rules against request {{(pid=81615) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 19 02:45:36.710405 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8dd6b-7192-4c29-8ac0-cb79c53121bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9243ebe24ae84157b74696c29bda3279', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '676e986ff972410d9f498800d313476f', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81615) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 19 02:45:36.711043 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8dd6b-7192-4c29-8ac0-cb79c53121bc service placement] REQUEST_METHOD: `GET` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 19 02:45:36.711319 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8dd6b-7192-4c29-8ac0-cb79c53121bc service placement] SCRIPT_NAME: `/identity` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 19 02:45:36.711590 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8dd6b-7192-4c29-8ac0-cb79c53121bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81615) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 19 02:45:36.713296 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.fernet_utils [None req-44d8dd6b-7192-4c29-8ac0-cb79c53121bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81615) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 19 02:45:36.724394 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8dd6b-7192-4c29-8ac0-cb79c53121bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 19 02:45:36.726085 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8dd6b-7192-4c29-8ac0-cb79c53121bc service placement] RBAC: Authorization granted {{(pid=81615) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 19 02:45:36.735760 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: [pid: 81615|app: 0|req: 1304/2611] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Oct 19 02:45:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 19 02:45:46.731445 dv-cinder-861491-4-iscsi devstack@keystone.service[81615]: DEBUG dbcounter [-] [81615] Writing DB stats keystone:SELECT=58 {{(pid=81615) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}